How can I observe myself when the self I'm trying to observe is this
pure observer?
Strictly speaking, no, there's no paradox in an entity observing itself. Such an ability can be embodied in all sorts of ways.
One example is a video recording device, where the camera lens can be extended using some sort of fiber optic tube and turned back to look at itself.
Another one is a computer program which takes other computer programs as input (for parsing or consistency checking purposes). There is nothing paradoxical about this program taking itself as input, thus "observing" itself.
More formally any information processing system capable of processing a representation of itself is capable of self observation. The system has to be able to store representations of other systems (in the video recording device's case, images, and in the computer program's case, descriptions of programs, in a person's case, mental images and language), and it has to be powerful enough, in terms of volume of data and/or symbols used for representation, to represent itself.
You are in a sense correct though. Although the act of self observation is not itself inherently paradoxical, it does lead to other paradoxes through self reference. Famous examples are Russell's set of all sets that do not contain themselves, and the liar paradox. These have been captured formally in Godel's incompleteness theorem and Turing's halting problem.
However, things get confusing because some of these same systems
(e.g.: Yoga) teaches that I am not the things I observe, but rather am
the one observing these things. However if this is true, this would
seem to lead to a circular situation.
You touch upon the fact that there is a hierarchical relationship between observer and observed, with observer being at a higher level since she/he does the observing, while the observed is the object of the observation, and is thus at a lower level in the hierarchy.
Such a person observing themselves would lead as you said to a "circular situation". Douglas Hofstadter calls this a "strange loop", any situation where a hierarchical structure twists on itself so that the top part of the hierarchy ends up connecting with the lowest level in the hierarchy. The circular situation you describe is just a two (or one) level strange loop.
The ability for self observation was used in interesting ways by philosophers.
- Aristotle mentions self observation as an affirmation of existence: "if one who sees is conscious that he sees, one who hears that he hears, one who walks that he walks and similarly for all the other human activities there is a faculty that is conscious of their exercise, so that whenever we perceive, we are conscious that we perceive, and whenever we think, we are conscious that we think, and to be conscious that we are perceiving or thinking is to be conscious that we exist... (Nicomachean Ethics, 1170a25 ff.)
DesCartes' "I think therefore, I am" is based on this: The existence of all external things can be doubted. Sensations, the world around us, other people, might just be an illusion. But there is one thing that we cannot doubt, and that is the act of doubting in itself. And for there to be doubt, there has to be an "I" that does the doubting, i.e. the very ability to observe ones own thoughts is proof that an "I" exists.
Hofstadter takes this idea and flips it on its head: The ability for
self observation is not the proof that the self exists. It is
this ability that leads to the emergence of a self in the first
place. Selves, "I"s, emerge as a consequences of information
processing systems developing the ability to store and manipulate
representations of themselves.
Although Hofstadter was the one who popularized the notion, he wasn't the first to propose it. Self-representational approaches to consciousness is ongoing topic of study among philosophers of mind.