Ok, I have a series of connected lines and arcs and I need to determine if one connected set is inside another connected set.

Here is what I mean:
{"name":"598600","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/9\/b\/9b9b66f2414a68a371789e002b2c4be0.jpg","w":551,"h":296,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/9\/b\/9b9b66f2414a68a371789e002b2c4be0"}

How do I algorithmically determine the smaller shape is contained by the larger shape?

Here is the information I have, endpoints of each line/arc, midpoints and angle of each arc, angle between two lines, minimum distance between two lines, angle between midpoint and line, minimum distance between midpoint of arc and line.

I have several "shapes" and some can have shapes contained in them and I need to group the shapes as separate objects.

I'm not sure I follow your second suggestion though I can get the intersection point of two lines as well.

According to my Introduction to Algorithms book from college:

Here's kind of pseudocode I would do:

You have polygons P1 and P2.

1. See if the first point in P1 is in P2. If it is, go to step 2, if not, go to step 4
2. Take the next point in P1 and see if it is inside P2. If it is, go to step 3, if not, neither polygon is completely inside the other.
3. Take the line segment from the point in step 1 and the point in step 2 and see if it interesects with any segment in P2. If it doesn't, go to step 2 (until all points in P1 are done), otherwise the test fails.
4. Try to see if P2 is in P1. Essentially, go back to step one and switch P1 with P2 in the order of testing.

I don't think there's any easy way unless you're dealing with exclusively convex polygons. And the best method I've heard of for dealing with that is checking to see if any one side in either polygon can act as a separating plane. If one can, they're not touching. If none can, it's a collision.

I don't see a problem with non-convex polygons (his example image shows both polygons as what I would call non-convex, i.e. not pass the rubberband test) but there would be a problem with complex polygons (some of the lines in the same polygon intersect). Perhaps there's a problem with terminology here.

[EDIT]

"Arcs" as a section of a circle or ellipse would be much harder indeed.

[EDIT2]
While playing with a new Firefox extension, I decided to Google the problem.

http://www.google.com/#hl=en&q=how+to+determine+if+one+polygon+is+entirely+within+another+polygon&btnG=Google+Search&aq=f&oq=how+to+determine+if+one+polygon+is+entirely+within+another+polygon&fp=jdOYMfF-0fs

What SiegeLord said. Split the the large shape into convex shapes to make it easier to check the points as 23 implied.

I second SiegeLord and jhukkson, but would add that the GLU tesselator is really useful if you need to break a polygon into a convex set.

If you want a really quick and easy test and have either a tesselator (set to in/out tesselation — the GLU tesselator can do that) or a CSG module to hand, maybe you could just compare the sum of the areas of the two polygons separately to the area of the two tesselated together? If the two totals are the same then they don't overlap.

Area of a simple polygon is trivial regardless of whether the polygon is convex, I refer you to question 2.01 of the comp.graphics.algorithms FAQ.

So do you just want a function that returns bool?
I'd imagine a function that first checks some simple thing and returns false (isn't inside) or continues with next check and returns false or continues with next more complicated test. And so on. Until you are sure to return true.

First test would be checking if any of the inner set vertices (or arc points) lie outside the outer set.

Second test would check for intersecting segments, like check every segment against every segment.

The second test is needed for this:
{"name":"598618","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/d\/ad6a792312c90c2603c0963b605cd440.png","w":794,"h":1123,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/d\/ad6a792312c90c2603c0963b605cd440"}

Well, can't think of a third needed test. Anyway, it's probably more clever to only test a against b and not also vice versa. Simpler logic. As I asked in the beginning, do you only want a bool return value? Like:

class polygon
{
    ...
    bool inside(polygon *another);
    ...
};

Then you do.

    polygon a(...), b(...);
    ...
    if (a.inside(&b) || b.inside(&a))
        ....

<edit />
Don't name it polygon, though

{"name":"598619","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/2\/a2a26dda6c35feed3d8a72eda72c41ff.png","w":1151,"h":296,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/a\/2\/a2a26dda6c35feed3d8a72eda72c41ff"}

Wouldn't it be sufficient to just do two checks then?
First, check if all points of polygon B are inside of polygon A. Then checks if all points of polygon A are NOT inside of polygon B (or respectively for D and C, see image above). In any case, the first check should always be a boundary box collision check and all the other checks should only be made if the boundary box areas collide.

edit: Ah, just saw Johans image with the orange and green polygon, so no, that would not be sufficient.

For C and D in the image above, their bounding boxes do not intersect (their areas do though) and there is at least one point of each lying in the other polygon, yet neither of them is fully inside of the other one.

edit2:Ah, sorry, misread what you said. You're not saying anything about bounding rectangles, which is what I automatically read when I read boundaries.