Rp1 will control how much current bypasses the cap and, therefore, how responsive it is to rapid charge events to prevent overcharging. The lower the resistance value, the more current it will bypass and the harder it will work to prevent the cap from overcharging. The trade-off is more power to dissipate in the P-channel MOSFET and the resistor. Based on the info in their documentation, it would seem that 100mA should be plenty for an application like yours. Since the Rds-on of the FET should be in the milliohm range, nearly all of the voltage will drop across Rp1 and you can calculate the value for Rp1 as:
Rp1 = Vcap / I
where “I” is the current through Rp1, or 100mA in this case.
The voltage across Rx2 determines the gate voltage of the P-channel FET. You want to scale it such that the P-FET is just beginning to turn on when the threshold voltage of M1 is reached. At that point, 1uA is passing through M1 and Rx2. The value of Rx2 will depend on the gate threshold value of the P-FET, so you may have to experiment a bit. In the reference design, they scaled it so that about 1mA passed through the P-FET when Vth for M1 was reached. Just a few millivolts above this, the current increased rapidly to 100mA, which clamps the capacitor voltage.
For this design, it is not as critical that the power resistors be right next to the FETs, as it is not a high-speed signal path. I would not go with the ALD810028, as it’s threshold voltage is too high for your caps. The ALD810027 or the ALD810026 would be better choices to protect your caps, with the 0026 giving you a little bit of a safety margin.
Notes on your design:
You are not leaving any margin for your current sense resistor. You will be passing 1W across it at the full 10A charge current, and the resistor you chose is rated for 1W. I would go with at least a 2W resistor in this case.
I don’t know where “CS” goes, but it looks odd to have a 10K resistor in series with “CS” as shown. Unless you have a pull-down resistor elsewhere, the voltage at “CS” will likely float when the phototransistor is off. If you place the resistor between the emitter (pin 4) of the phototransistor and ground and tie “CS” to the emitter also, then “CS” would be low when the phototransistor is off and high when it is on. If you want reverse logic, connect the resistor between 5V and the collector (pin 6), tie the emitter (pin 4) to ground, and connect “CS” to the collector (pin 6).
Also, a minor note: you have the connections for the P-channel MOSFETs, Q10-Q12, correct, but the internal drawings are wrong. You have them drawn as N-channel MOSFETs instead of P-channel MOSFETs. They should look like this: