The Airbus A330 from Perth to Kuala Lumpur experienced problems about 90 minutes into the journey.
It landed safely, with Perth Airport reporting a "technical issue".
"The pilot identified technical issues, turned around and returned," an airport spokesman told AFP, adding that emergency crews were put on standby but not needed.
The Malaysian low-cost carrier said only that "flight crew are taking precautionary measures to check the aircraft".
It was the second scare involving an Airbus A330 in Australia this month, with a China Eastern plane making an emergency landing in Sydney after a huge hole appeared in one of its engine casings.
Terrified passengers on that flight described a very loud noise soon after it left Sydney for Shanghai. No one was hurt.
The West Australian newspaper cited passengers on Sunday's flight as saying they heard a bang and then the plane started shuddering.
"You could tell by the cabin crew's reaction that it was really bad," said Sophie Nicolas, who said she heard a small explosion from the left wing. "It was terrifying."
Another passenger, Brenton Atkinson, told the Australian Broadcasting Corporation the whole plane started shaking, far more than standard turbulence.
READ MORE
