If you look in the service or parts manual you will see there are 2 seals on the shaft, one for the water, the other for the oil. The weep hole is from the area between. Guessing you know this already. Most important is that you're not getting water into the transmission. Maybe drain the oil and check for water. If there is water in the oil I would change the seals. If no water then I would just make sure you monitor the situation close. Either way you should plan to change it at some point. If your bike has a wet clutch you can inspect and replace the clutch plates as well while in there. You'll need to replace a few gaskets while doing this work. Water pump cover and clutch cover, all available. Water pump shaft and seal kit also available. The plastic gear on the water pump has been known to crack but I think Andy will say they generally don't break. T2 racing makes a metal replacement..if you get to that point.
I worked on a friends Moriwaki MD250 last year and we had same issue, the bike has a 4 stroke dirt bike motor. The set up is very similar to NSR. In the Moriwaki case, the water pump shaft had worn and was actually causing the leak out the weep hole, no water into the crankcase. I replaced shaft and seals. We also needed new clutch basket which what prompted me working on bike and I found the water leak while inspecting things.
If I remember correctly, you don't need to pull the engine to replace the seals. But it does take some effort as the seal gets destroyed when removing it.