The Ancient Civilization were a race of proud, magically adept beings that destroyed themselves by overuse of Nightmare Lights, causing the periods of magical flux and possibly unleashing Shadow Creatures which helped in bringing its ruin.
It was this civilization that produced the Clockwork Monsters, which can found swarming the military biome, suggesting they used them as mechanical soldiers. Maxwell found them and adapted them for his own use. It seems that they were also religious as the Ancient Pseudoscience Station is referred to by Maxwell as a place they may have worshipped at. Some of the magical items prototyped from the station show that they were very skilled in the use of magic, as they used staves cable of summoning miniature-stars, to short ranged teleportation, to armor cable of producing short lived force fields. The ancients also appear to have amassed a large wealth of the now rare material, Thulecite, as many of their artifacts have either been constructed entirely of it, adorned with it, or contain trace amounts of the material.
The ancients also seems to have a powerful Military, as their remnants are littered with Clockwork remains. The ancients were also protected by a powerful guardian, but even it's great strength helped little against the shadow creatures, and was transformed into the beast that can be found in the deep labyrinths of the ruins. After the once great race had ended, all that remains are their technology, relics, and the aftermath of the nightmare abuse, the Nightmare Cycle.