How was a day determined before the sun?
Evidently the Earth was present before there was light. "And the Earth was without form and void; and darkness was upon the face of the deep." When did God create the Earth? And related to this, what was the light of the first day? How was a 24-hour day determined before the fourth day?
Many people take Genesis 1:1, "In the beginning God created the heaven and the earth," as an introduction to the story of creation. As a result, it looks to them like God started with some sort of formless Earth at the very beginning of creation, which is not the case at all.
Creation was ex nihilo, which means that what God created He created out of nothing. He did not start with a formless and void material which would become the Earth. Genesis 1:1 is not an introduction statement but God's first act of creation. In the beginning God created all the matter in space and especially the Earth. After this point of first creation, ex nihilo, the matter and energy were present, but they were not necessarily organized into working systems. Nevertheless, the very creation of matter and energy did establish some relationships which we can understand today.
The statement "In the beginning" indicates that time began with the initial creation of matter and energy. It suggests that these two may be tied together. And, in fact, today physicists describe time as a function of matter. This means that matter and time are fundamentally tied together in some way.
But as yet, the material of the Earth and the material in space were evidently not consolidated into any sort of working system. The very first item of business in God's creative action of making this material into working systems was the creation of light.
You ask, what was this light? The fact of the matter is that there are so many possible choices for what this light could have been that we cannot say for sure what the source of the light was. The text may give us some hint of the characteristics of this light. First, this light was divided into light and darkness, giving day and night to the formless Earth (verses 4 and 5). Second, this light was evidently not adequate to support plant life on the Earth, requiring the creation of the sun on day four (verses 6 and 7). These statements give us hints about two properties of this initial light. Whatever its source, it seems that it was either not bright enough to support life on Earth or it was not a full-spectrum light such as we receive from the sun.
The creation of the sun on the fourth day, in the context of the important part that we know sunlight plays for life today, seems to suggest that the initial light of the first day was not sufficient to support life.
It is, however, anybody's guess what the source of this first light was. The only property of this light which Scripture tells us about is that it was divided in such a way as to produce day and night on Earth. It may have been the glowing of the material which God used to form the sun on the fourth day.
Another possible source for this divided light could have been the material which God used to create the stars or planets. Or He may simply have created a specialized, diffuse glow, using the free hydrogen in space.
Some have suggested that the source of this light was God Himself. However, Scripture describes God as the source (in the sense) of uncreated light. This clearly was created light.
You also asked how a 24-hour day was determined before the fourth day. Genesis makes it clear that this initial light was divided, giving both day and night to the Earth. We should also remember that the passage of time is not linked to the sun. We measure time in days even when we cannot see the sun. Submariners who spend months under the water, often weeks at a time in the darkness beneath the polar ice cap, have no difficulty measuring a day. The same is true of astronauts who see many "days" each day.
Surely, then, if we can measure time without the sun, it is not unreasonable to expect that God can also measure time without the sun. Remembers, this is His account of what He did.