Short answer
A pot that dries out in under a day usually has very little soil left in it, because roots have filled the container. The other two common causes are a potting mix that dried out completely and now repels water, and garden soil used in a container instead of a potting mix. All three are visible when you slide the plant out.
Listed in the order they are worth ruling out. It is normal for more than one of these to be happening at once, so work down the list rather than stopping at the first match.
Soil that dries out in under a day usually means there is not much soil in the pot any more. A root ball that has filled its container holds almost no water, so everything you pour in runs past it.
What to check first
Slide the plant out of the pot. If you see a dense mat of roots circling the outside and very little visible potting mix, the container is full.
If a peat based potting mix is allowed to dry all the way through, it becomes difficult to rewet and holds less water than it did before it dried. Water then runs down the gap at the side of the pot and straight out, which looks exactly like fast drainage.
What to check first
Water it and watch the bottom of the pot. If water appears within a couple of seconds, it went down the sides instead of soaking into the mix.
You said the pot was filled with soil from the yard. Field soils behave differently in a container than they do in the ground, and the result is usually a pot that is either soaking or bone dry with little in between.
What to check first
Squeeze a handful from the pot. Garden soil holds together in a dense clod and stays muddy. A potting mix is light, springy, and full of visible white or bark coloured pieces.
Still not sure
This is exactly what your county Extension office answers, for free. They are the local horticulture experts for your area, they know what is going around this season where you live, and they will look at a photo.
Find your local Extension officePollinator Patch gives you watering and seasonal care for the plants you already own, written for your region rather than for gardening in general.