There is the adage that you are the average of the five people closest to you. In writings about how to be “successful”, the advice is usually then to cut out the losers and get closer to people that are more successful than you.
If you have a truly toxic influence on your life then I agree, best to distance yourself from them. I have had to do this myself, but in hindsight, I realise that it was my problem rather than theirs. However, the ruthless culling of true friends based on their success is not something that should be encouraged.
Ask yourself, are they really holding you back, or is it just you?
Remember, you don’t have to listen to what other people say. If you cannot distance yourself from negativity and criticism then you are probably not going to have the spoils of the “successful” that can.
Also, remember that you are one of the five too. How much are you trying to pull the others up? Those that have been there for you in the hard times.
As Gary Vee says, there are two ways to build the biggest skyscraper in town. Afford this thinking to your nearest and dearest before you bulldoze everything around you.