Today I found out that pipe cleaners (like the kind used in arts and crafts) work wonders in cleaning tight areas like inside the plastic valves of my playtex no-spill sippy cups, and inside the stem/spout as well. I also assume they can't be used as a bottle brush if you wrap them around a piece of wire. Or just use them normally like I did but sticking them in and sliding them around with a little bit of pressure.
Either way it really removed whatever this black stuff was from one of my cups. I am thinking it was left over pulp from some Simply Lemonade that was just rotting. Since it wasn't bunched up in a area like mold/mildew normally might.