Doesn't seem stupid to me. Unless you verify it's not something simple, you could waste lots of time looking at more complex possibilities.
That said --- i was actually hitting save, and the contacts would disappear afterwords.
Also, I found what my issue was -- it is posted above.
It was due to some very non-intuitive design, that will hopefully get updated to act more intelligently.