S
You notice that you now have access to 6 availability zones in us-east-1, where you previously only had 4 or 5.
Adding capacity is the reason that new availability zone is now available.
Because of the way the spot market works, there is no way to know what the highest winning bid was, at any given time... because the current market price is based on the lowest winning big for each round. That sounds counter-intuitive, but here's the logic: given n available instances in the spot market, the system identifies the highest n bids. These bids are the winning bids. The lowest of these winning bids is the price all of the winners will pay, and becomes the new current market price.
Each time the market shifts (new bids arrive, or active apot requests are canceled, or on-demand instances are started/stopped, presumably resulting in the available spot pool growing or shrinking), the system re-evaluates all the bids. If you are no longer a winning bidder, your instance will be terminated, but if you're a new winner, you'll get the instance you've been waiting for.
Based on what's publicly known, we can conclude these things:
the current market price is the lowest bid among all the current winning bidders, assuming there are any winning bidders
you will not get a spot instance right now if you bid below the current market price, but you'll get one if/when the price drops below your current bid.
Therefore, when the spot price is at maximum, there are no unsold spot instances. We can logically conclude that this is necessarily true, because:
a single winning bid that is lower than the maximum would cause the market price to be lower than thr maximum; therefore, it defies logic that there could be any spare capacity, because this implies that all current winning spot instance bidders made outrageous bids when they could have obtained the instances for substantially less money either by placing a smaller spot bid or by using an on-demand instance
it also defies logic that AWS would artificially inflate the price above the on-demand price, because there is no rational reason for someone to pay more in the spot market when the could use an on-demand instance.
We can further conclude that one of two things has happened to cause this, both of which have the same net result:
there is no spot capacity available at all -- if this is true, it makes sense that the indicated price would stick to the upper maximum value, because you must bid at or above the current price to get an instance now. The actual market price in a situation where there is no capacity at all would be more properly called NULL because there is no price at which you can obtain a spot instance. Or, you might say the current market price is infinite, because regardless of how high you bid, you can't have a machine that doesn't exist. Some might naïvely say the current price is $0 but that is definitely incorrect and would send the wrong market signal. Where this leaves us is that the most logical representation for the price in a zero-capacity condition would be the maximum/ceiling price. This gives a clear market signal to any sensible algorithm -- don't bid in this market, the value is not there; or
there is some capacity, but it's all claimed by people who bid the maximum possible.
Which of these two conditions is actually true doesn't matter, because the net impact is the same -- no available capacity.
If that second possibility doesn't make sense to you, that's because you are a rational person. There are always people who try to game the system.
Instances in the spot market are ephemeral, since the market shifts are outside your control. I've had spot instances run uninterrupted for many months on end, and I've had them cycle up and down many times over the course of one day. This is part of the market.
It is possible to improve the chance of a spot instance persisting by placing a high bid... but it is not possible to guarantee that a spot instance will keep running, regardless of how high you bid... because even if all the high bidders colluded with each other and bid the exact same high price, the capacity is finite, so if all current spot instance capacity is sold, not all of the high bidders would have machines allocated to them.
However, there's always that guy who thinks he's smarter than the rest of us, bidding far too high, presumably on the assumption that his instances are likely to continue running, and that since price spikes are transient, he expects only to pay the high price for only a short time. If there are such gamblers in the market, then they would have the same effect of pushing the price to its limit when capacity is constrained. This becomes clearest when you consider the fact that if there were only 1 available machine, then only 1 person needs to place an outrageous bid, to set the market price unreasonably high.
The maximum spot price bid is now 10x the on-demand price. This has not always been the case. The logical conclusion is that there are indeed those who bid outrageously high, because they are trying to game the system and use spot instances for a persistent workload, so they bid high to try to keep those machines running. It's a flawed strategy, but it does explain the prices hitting the ceiling and sticking there when capacity is limited.
