Tl;dr if old weight = x, old fraction of wax = m, fraction of candelilla = k, then new weight = 2x / ((1 + m)(3k+1) - 2m).
It depends on a lot of details. These are all organic materials, so this isn't an exact science, and the closest exact science (chemistry, specifically thermodynamics of phases) is still maddeningly complicated. There are materials out there that can have a lower melting point at both 90-10 and 10-90 mixtures than 50-50 mixtures, for example, and they're not even complicated mixes of various-length variously-branching various-impurity-containing carbon chains themselves, which waxes are.
The half/double rule of thumb is just that, a rule of thumb. I don't know how much non-wax material the recipes use, but say it's 3 ounces, and 1 ounce beeswax. The beeswax is 25% of the weight then. If you use half the amount, it's now 3+0.5=3.5 ounces total instead of 4, so the wax is one seventh, 14ish%, not 12.5% as you might expect, and if you use double, it's not 50%, but 40%. So multiplying the _weight_ and multiplying the _proportion_ are always going to be different, but the difference is small if the proportion is small to begin with.
The simplest model would be that there's a "hardness level" you're trying to achieve, and either the materials add to the hardness level, or average to the hardness level along with the hardness of the rest of the recipe. Otherwise everything is linear. Then there are two cases: A, the rule of thumb didn't take the difference into account, so add weights, and B, it did, so add proportions. If the fraction of wax is tiny, just ignore B, but if it's something like half of the ingredients, don't.
To the math. Let's use x = original amount (weight) of beeswax, w = new total weight, y = candelilla weight, z = bayberry weight. So in your case you set y=0.6w, z=0.4w. In general, if you want to use a fraction k of candelilla, y=kw and z=(1-k)w. For case A, y has double effect and z has half effect. c is the weight of the rest of the recipe. We're looking to solve for w.
In case A, x = 2y + z/2. Replace to get x = 2kw + (1-k)w/2, or w = 2x/(3k+1). For k=0.6, w = x/1.4 ≈ 0.714x. Here's a table:
Candelilla | Bayberry | total mixture ounces to equal 1 ounce beeswax
:--|:--|:--
0% | 100% | 2
10% | 90% | 1.538
20% | 80% | 1.25
30% | 70% | 1.053
40% | 60% | 0.909
50% | 50% | 0.8
60% | 40% | 0.714
70% | 30% | 0.645
80% | 20% | 0.588
90% | 10% | 0.541
100% | 0% | 0.5
Case A is simple enough to interpret: you mix something with hardness 2 and something with hardness 0.5 to get something with hardness 60% \* 2 + 40% \* 0.5 = 1.4, so you need only 1/1.4ths of it to replcace something with hardness 1.
In case B, x / (x + c) = (2y+z/2) / (y + z + c). Replace to get 2x / (x + c) = (3k+1)w / (w + c), or w = 2xc / ((x + c)(3k+1)-2x). Or define m=x/c to be the fraction of wax by weight in the original recipe, then w = 2x / ((1 + m)(3k+1) - 2m). For k=0.6, w = 2x / (2.8 + 0.8m). So if beeswax is, say, 25% of the original recipe, m = 0.25, and w = 2x / 3 ≈ 0.667x, but if it's 75%, then w = 2x / 3.4 ≈ 0.588x. Not a lot of variation, but it's there. If you assume m=0, you get case A back.
Finally, keep in mind this could all be wrong, because the assumptions (linearity etc.) aren't really true. Experimenting is worth it and probably the only way to get reliable results in this case. Also there are more dimensions than hardness.