I want to make my project rechargeable but don't have a deep understanding. Would it be ok if the load was on (ON/OFF SWITCH closed) and the battery was charging at the same time? The load draws about 60mA.
I use a MCP73831 LiPo charger set for 100ma charge current with a MCP1252 boost to +5v and that's how mine is configured. No problems.
I use the MCP73831 as well, but if the OP wants a module-based approach, both Sparkfun and Adafruit have boards available. 3V isn't a 'standard' load (most target 5V), so you'd still need a regulator for that. For 60mA I'd be happy with a simple LDO linear regulator, to avoid the complexity of a switched circuit.
The best approach would be a power-path controller IC, maybe something based around the BQ24075. The problem with simple battery chargers is that, as soon as the battery is fully charged and the charge current is switched off, the battery starts to discharge and the cycle starts over again. This is less of an issue if you can remove the load whilst the battery is charging, but the use-case may not allow for this.
Does the chosen battery have a built-in protection circuit ?
Would it be ok if the load was on (ON/OFF SWITCH closed) and the battery was charging at the same time?
It depends...a dumb chip might not notice, a smart chip might get its charge controlling confused, thinking it is delivering xxx ma to the battery, without knowing xx ma is going elsewhere. An extra-smart chip might be able to measure this loading & take into account. However, at your low loadings it might just be a drop in the bucket & of little consequence (depending on the ratio to the charge current---if it is 1amp vs 100ma, 60ma may or may not matter).
Edit---I noticed your diagram has a proposed chip...looking it up I found:
It does have a protection circuit. Overcurrent, overvoltage and short circuit protection.
Hmm I see the problem. Since the battery does have an integrated protection circuit that protects from overvoltage, does this solve the overcharging problem? Should I still try to avoid this design?
The problem, as @avrcandies' edit points out, is that the TP4056 gets confused by the current drawn by the load. Even if it doesn't damage the battery, it will upset the charging cycle, which may either never complete, or finish early. So you'd never know if the battery was fully recharged.
Read the description of this module to see why something like the MCP73831 is better: https://www.adafruit.com/product...
Non the less it should be a safe bet, no? Considering my low load and the fact that the battery in question has an overvoltage protection on it. I assume there will be no flames, right?
I assume there will be no flames, right?
Put one in your pocket for a few weeks & see how it goes.
jgmdesign 
I used an MCP73871 for a project two years ago . I also used a Turnigy LiPO battery pack that included the Over Voltage/Undervoltage protection built in. What is great about this combination is that the MCP was a full blown(no pun) charge controller AND battery status monitor. I was able to connect it to my AVR and get battery status from the device through the RF link.
THis piece of kit has been running flawlessly since delivery. No smoke, fires unhappy customers.
It was a little more $$ wise to use, but better than the alternative.
Jim
There is a cheapish module based on the MCP73871 available: https://www.diymore.cc/products/...
Microchip have a couple of eval boards as well: https://www.microchip.com/wwwpro...
awneil 
The best approach would be a power-path controller
Absolutely - that's the key term you're looking for.
