A square circle would be a shape or object that is both square and circular. Philosophers, especially working in philosophy of language and metaphysics, use the phrase "square circle" as an example of a contradiction in terms, that is, a phrase (as opposed to a proposition), two parts of which describe qualities that cannot both exist in the same thing at the same time.
It is also often used as an example of an "impossible object." Probably the most interesting philosophical question about square circles, and other such "impossible objects," is whether they enjoy any sort of existence or being. The 19th century German philosopher, Alexius Meinong, famously held that while such objects obviously do not exist, they nevertheless enjoy a queer sort of "being." Other philosophers have held that "square circle" is literally nonsense, that is, lacks any significance or meaning. An interesting feature of that position, however, is that it denies significance despite the fact that the words "square" and "circle" do individually have meaning, and we can say what their truth conditions are (i.e., necessary and sufficient conditions of their being true).
Clever schoolboys may point out that a three-dimensional shape may be square on one plane, and circular on an orthogonal plane. To their disappointment, philosophers stipulate that that does not count.
Note that this does not have to do with the geometrical problem of squaring the circle, that is, constructing a circle that has the same area as a square using only a compass and straightedge.