Click File -> Options -> Calendar.

Click on Add Holidays... button

Select the Country/Countries for which you want the Holidays added to the calendar by clicking the checkbox next to the appropriate country and click OK.

Outlook will confirm that the holidays have been added to your calendar via a pop-up window notification.

Removing Holidays

If you want to remove all holidays from Outlook calendar, click View > Change View > List.

Click View > Categories to sort calendar items by category.

Right Click on the Holidays tab and select Delete.

You will get a warning message. Press OK.

Holidays will no longer appear in your calendar.

Removing Specific Holidays

Click on the Calendar at the top View -> Change View -> List

Select the holiday(s) in the list which you no longer wish to have displayed on your calendar, then right click on the event and select Delete or hit the Delete button on the keyboard.

Note: You can delete multiple events by holding down the Ctrl button on the keyboard while highlighting each event in the list.