A computer to be used by children is at best a dedicated PC. You will not have to worry about protecting your data or other resources. There is simply nothing there for the child to erase or reconfigure. Whatever you do, never, ever let children use a computer you use for work, no matter what their age is. Accidents happen and you will be the only one to blame for allowing it. If necessary, use a USB key or external hard drive to boot from. An 8GB USB key is enough to put the OS and plenty of software on. A key like this is very cheap and certainly a lot cheaper than lost data.
If you use the free and open operating system Linux, you can get away with older hardware. I have recently used a PIII of 700MHz with 384MB of RAM with great success. It ran all the educational software without a hitch and even ran Flash at acceptable speed. This machine is an exception though but in general I can say that any Pentium processor with at least 512MB of RAM is enough to get a more than acceptable speed. This puts the Kids PC in the category of “old PCs too slow for a modern Windows OS”. Many people throw these PCs out the door so you can easily get a PC like this for free if you ask around.