Hello MT APO,
Looking at the example that you had provided, I could guess that you have the following settings in check instructions
1) Product Allocation check - 1st Check
2) Product Availability Check - 2nd Check
For me, the example that you have provided is clear and system is working fine as expected as per the above settings. Please remember that Rule Based Availability is more of a "ATP" functionality but not "Allocation" relevant. Net, you can consider RBA as one of the sub processes of Product Availability though it plays a major role.
The allocation figures that you maintain in the system are actually not the accurate stock figures. They are virtual figures that users put in place to cap your availability check. In the above example quoted by you, the first check is actually acting as a filter to pass on the filtered quantities to next step. So, if your allocation says 0, then the total confirmation would be 0. Although I understand that there is an allocation limit on the substitute product. If you think widely in business terms, if the users have decided to do the substitution between two products, to the best of my business knowledge, I would think that the allocation limit should also be more or less same between both the products because you consider them as one entity or one group to substitute each other.
Now, next thing would be to see if you can make Product Availability as first check and Product Allocation as second check. Even in this case, I think ( but I am not confident 
) the 2nd check will cap your net remaining requirement to 0 since there is an allocation of only 0 pieces. You may test this part and educate the users accordinly.
I hope I have been of some help in answering your question.
Thanks & Best Regards,
Babu Kilari
SCM APO Consultant