In fishing there is a thing called "match the hatch". What this means is simply match your bait to what the fish are eating at that time at that location.
Your shrimp and worms do not normally drift around. They are normally found on the bottom. So stop using the float or change the bait to one that does drift around. Even a fish shaped soft plastic would do better under your float than a dead shrimp...me thinks.
Fish hang around structure. Use your fish finder to look for structure. If the bottom is flat, the fish will be far and few. Look for anything that stands out. Rocks, mounds, holes etc.
When you are new to fishing it is temping to cast out as far as you can, or if you have a boat, to go out to deep water. This is a mistake. You will catch more fish in shallower water close to the shore, especially if there is structure like a jetty, over hanging tree or moored boats etc.
Do you ware perfume and make up? Yes? Wash your hands with aniseed soap before you handle your bait or any of your gear (rod, reel, hooks etc.) and before you go fishing. Any good tackle store will have aniseed soap.
itsaboat mate...Life is just a boat and then ya marry one !