Am I washing my glasses wrong??

I agree it’s your towel. The towel below will solve your issue.
https://www.wine.com/product/riedel-polishing-cloth/126100?state=CA&s=GoogleBase_CSE_126100_PS106__type_WineAccessories&showpromo=true&promo=PS106&region_id=000008&utm_source=google&utm_medium=cpc&utm_term=&utm_campaign=Google_Shopping_Smart_Wine&gclid=EAIaIQobChMImpS1s8eB9QIVeT2tBh1ujACQEAQYAiABEgKeFfD_BwE&gclsrc=aw.ds#

I’ve switched from linen to Grassl microfiber from Chris Freemot. Linen get wet too fast and wet linen is heavy. My Grassl glasses hate weight/resistance. Different strokes.

Soap and hot water = clean. Regardless of what you are cleaning. Proper rinsing leaves no residue. I personally believe that residue left behind is from not getting glass clean enough in the first place.

Microfiber comes in dozens of different types, just like cotton towels. Diff weaves, grams per square inch, length of nap, and different poly blends. Not that we need to go that into depth.

Obviously, the towel needs to be clean to not re-contaminate the glass.

The Grassl microfiber has zero nap. Zero streaking. Zero spotting.