Didn't vote for me, won't work for you, says BJP MLA who lost Lok Sabha polls