Staying Curious with Clean Language