I myself use a big backpack for my diaper bag. I use cloth diapers for several reasons, and they do take up a lot more space than the typical disposable one would. I carry two with me, just in case I'm out for an extended period, along with the usual supplies.
When I'm in a store or restaurant and have a problem, {a messy one, wet ones can wait} I'll go back out to my car and get it and come back inside to change in their restroom, unless I'm close enough to home of course. I've never been questioned as to why I'm carrying it into the store, probably because I'm almost 50, and I walk with a cane. But still, unless you look like some kind of creep or shoplifter, you probably won't be asked either.
For example, last night my wife and I went and saw the movie "Four Christmases" at a nearby theater, and I carried my backpack right on in, like many times before, and again, nothing was said. I think that if you don't make a big deal out of it, and you don't look nervous or jumpy, most people wouldn't give you a second glance.
But I've had to do this now for almost six years, so I'm pretty much comfortable with it. I've accepted that it's going to be this way for a long, long time to come, unless the doctors can figure out what nerves were damaged during my second back surgery and can fix it.
But any kind of bag that you're comfortable with will work. I used the backpack because it's what I had laying around the house, and it's easy to carry. For me, a gym bag would just look out of place, lol.