Startups that are just getting going and need something to help them manage their finances can get by with something simple, such as Quickbooks, Peachtree, or Freshbooks. Many startups (including some that are building hardware in house) spend their first several years managing their finances on these types of platforms, and they use spreadsheets for inventory management. This is not scalable of course and only works before the startup gets to scale.
Once a startup gets to a level of significant scale, it may be time to start thinking about transitioning to an ERP (Enterprise Resource Planning) system. The acronym "ERP" strikes fear into the boldest of hearts because without fail, they are very expensive, very slow and difficult to implement, and not at all easy to use. That said, as the business grows and scales, the financial backbone of the company needs to be sound and at some point, companies will need to transition to an ERP system. Example ERP systems range from hosted platforms like Sage and Netsuite to mega enterprise solutions like SAP.
ERP implementation can take months (at least 6 months for hosted services like Sage and Netsuite and much longer for SAP) so startups should really stay clear of those systems until they have the scale in operations to justify the effort.
The type of companies that need to transition sooner than later tends to be hardware startups, because they will start to outgrow their spreadsheets for order and inventory management. The general rule of thumb is that something like Quickbooks is probably adequate until the startup reaches $10-15 million in annual revenue, at which point it may make sense to switch to an ERP system like Netsuite.