You could pull a luma key for the dark areas that encompasses the faint lights but nothing brighter. Use a comp node to multiply the plate by this. Blur both comp node result and alpha the same and then dived the two blurred results.
Basically the object obliterator on the darks.
Comp back using your luma key.
Matte blur i think it is also called.