There was, indeed, a much larger town here at one point! There are a couple different sketches of the town in a few of the books, and while most are of Wellstone more or less as it is today, there is one, labeled in faded, unreadable writing, that indicates a sprawling, thriving town with many more buildings. This book is the oldest of the lot and the pages with those sketches are quite decayed, but the buildings that can be made out are:

-the last building on the edge of town, just before the stretch leading to the diner: a gleaming stable for many horses. This corresponds to a ruined pile of wood at the moment with a few rotting timbers left.

-between the bathhouse and the General Store used to be another wooden building with the sketch of a little marquis with letters out front. It looks very like an old theater, and a large sketch a few pages later confirms that it did used to be a theater-slash-dance hall.

-in what is now the desert between the town and the graveyard are many little buildings that look like personal houses.