There are many forms of being haunted. Some are mystical and others can be psychological. Aziz is haunted by his arrest, the bitterness still clouding his judgement. His overactive imagination overpowers him and leads to a lot of his less than savory choices. He’s made assumptions, had suspicions, and relied on his own beliefs for so long that this false narrative he’s created has one upped him. Still plagued by the damage dealt to his reputation, Aziz holds on to that bitterness and his past actions haunt him in part 3 of this story.
Until he can let go of his assumptions, or officially have them proved false, such as learning that Fielding did not marry Adela, he’s weighed down by his bitter attitude, his past mistakes, and his assumptions. Of course, when he does learn the truth, he’s freed of this haunting bitterness but still torn between his new range of emotions that he experiences when he learns he’s embarrassed himself by making assumptions in the first place. I suppose he will always be slightly haunted by the situation if not mostly free from it in the end. That does not mean neither he nor Fielding do not suffer from making the same mistakes nor that they do not make new ones. Again, I suppose that if you cannot learn from your mistakes they will always truly haunt you. These two characters are victims of circumstance. They can try to have new, well intentions, but they are going to have to truly work for them, and their are many outside circumstances waiting to sway them. It’s like being haunted by your own cultural differences.
There are multiple possibilities for the concept of haunting in this text. One may idea,as you said, does seem to stem from Aziz and his rage. It is impossible for him to let his anger and frustration go as he ultimately haunted by his experiences. Coinciding with this, each member participating in this travel to India was looking for more of a concept rather than a location. Their desire to see “the real India” cannot be fulfilled as they left to experience what India now has to offer, not India of the passed or India dressed to meet their expectation. As another classmate mentioned, a lot of the tension building up in this book comes from racial and societal issues that rise and linger, continuing to haunt our characters and break down their expectations.