If you paint or you play an instrument people can tell whether you are any good or not, also these are interests you can legitimately claim to be pursuing purely for your own pleasure. If you are writing a novel you are doing so in the hope of being published, if you are writing a screenplay it with the intention of getting it filmed; unless you have a track record of success to point to, then the only way anyone can tell if you are an undiscovered genius or a deluded wannabe is to read the damn thing. So unless you carry round copies of your manuscript to hand to people, and are prepared to stand over them while they read it, then in the interests of avoiding social awkwardness literary aspirations are perhaps left unsaid.
Having said that, you are not going to get anywhere without self-belief, so if pursuing your dream matters to you more to you than the judgement of casual acquaintances, then wear your "I am a writer" hat with pride.