In item #5, where print uses squares to indicate a missing something in a mathematical expression, we’ll use the Nemeth square shape, as you say. With the goal of giving the reader consistent symbols, we’ll also use the Nemeth square for the other square, after “value of”.
The shapes that precede each answer choice are not part of the meaning of the text. I would omit them in braille or use a transcriber-defined shape for these squares. (The “transcriber-defined shape” is explained in the Rules of UEB, and the Guidance for Nemeth within UEB says it can be used both in UEB and between Nemeth Code switch indicators.) Whether you omit the squares before answer choices or transcribe them using a transcriber-defined shape, you’ll be giving the braille reader the same thing at the beginning of every answer choice.
Thank you for the questions! Please do let us know if you have more.
–Kyle