How do I know if there is an update available?

The detailed change history of each item can be publicly accessed in the changelog tab:

One example: https://mui.com/store/items/minimal-dashboard/#changelog

The store doesn't provide a notification system for new updates yet. Instead, you can have a quick glance at what the latest version is, and when it was last released.

Once you have identified an update/upgrade you would like to use, you can follow: How do I get access to an item I purchased?