Twitter – Reply to tweets don’t appear in the tweet box

If you delete a tweet it disappears for ever (although in some cases they may appear in Twitter search for a little while after)

Twitter and any clients using the streaming API will remove the tweet from a users stream almost immediately. Other clients using the REST API will remove the tweet once the timeline is refreshed.

I'm not 100% sure on what would happen mid-reply but I would have thought the reply will be sent successfully as the id of the now deleted tweet will have already been attached to the new tweet. I guess it would depend on how each client handled incoming and outgoing tweets. In any case the worst thing that would happen would be an error message being to displayed to the user.

Deleted tweets will disappear from a conversation chain.

I'm not sure if deleting tweets is frowned upon but if the information you have posted is incorrect or you no longer want to tweet it then it's entirely up to you as to whether you delete the tweet or not.

These tweets are always public. Anyone can see them anytime. But Twitter website is not built to show @B's replies to @A on the home timeline if they are not following @A. But there are Twitter clients that does it. I remember TweetDeck having this feature. It'd show the followers of @B all of his tweets and replies to @A even if they are not following @A.