Adding a cell in a sheet with a specific column number requirement that pushes cells to the beginning of the next line

I have a sheet that I need to only be 5 columns. I use this as a teaching calendar. Sometimes I need to add a cell in the middle if there is an activity that will add an extra day into my calendar (or what I planned for one day, actually takes two days). However, adding a cell only pushes the row to the right and adds a new column.

Instead, I need it to keep to only 5 columns and instead move the cells to the beginning of the next row instead.

Is this possible?