In the name of Allah, the Inspirer of truth
Haj is necessary for a woman if she has a mahram [unmarriageable kin] or husband to accompany her on the journey. This means that she has to have a mahram or husband who is willing to go along. If a mahram is willing to go along but does not have sufficient funds for himself to make the trip; or has sufficient funds but does want to spend of it, then she is obliged to pay his expenses too if she has the means to. If she does not possess the means of paying his expenses, then haj will not be necessary on her (Radd al-Muhtar 2:145, Muallim al-Hujjaj 84).
In your case, haj would be obligatory on you if you brother is mature [or close to maturity but is sensible and understanding] and is willing to accompany you, and you have the means to pay for his expenses as well.
Regarding the sin in delaying it, it is stated in the al-Durra al-Mukhtar that it is prohibitively disliked to delay it once it becomes necessary upon a person. However, according to the stronger position if the person performs the haj prior to dying the sin is lifted (Radd al-Muhtar2:140)
