Woocommerce List categories with Image thumbnail

I installed WooCommerce 4 and I am trying to list out all the categories with thumbnails. Can I achieve it by a plugin or widget? Or I have to write the codes by myself?

The thing I want to achieve is similar to this:

enter image description here

Thanks in advance!