He was scared to hide in hunger and illness after being shunned by everyone