The expansions have narrow design goals, and the core set is there to bridge the expansions with toolbox cards that are needed for many decks. If a card does not fit into an expansion's "thing" it can be put in a core set because the core set can have a cute little theme going, but it can also have a bunch of toolbox cards that are in the set for deckbuilding reasons. Without the core set, deckbuilding devolved into mostly block-only decks and cards needed to police broken strategies were left out (they could have been in a core set). This caused many bannings in recent Standards and, generally, it caused some players to play less Standard, or no Standard at all.
The core set also allows expansions to explore their "thing" at a deeper level because the cards that help that "thing" now don't need to be in the expansion. They can now be in the core set before the three other yearly sets, or the one after (or in both). Kaladesh, for example, could have had more energy cards (not that many people would have loved that, this is just an example), but many of those additional energy cards couldn't be put in the set because, in the absence of the core set, all of these toolbox generic cards had to be put in Kaladesh. Effectively, every set beginning with Battle for Zendikar is really 3/4 expansion, and 1/4 core set, even if the cards that would have been in a core set have the expansion's symbol. Examples: [[Murder]] in EMN and [[Naturalize]] in Rivals of Ixalan. Both are typically core set cards.
