So for a WAG; I would have your trains operate at "restricted speed" (usually about 20 mph) or less, this is also implies a rule specific set of contingencies and heavier trains may go significantly slower.
To attempt to better answer this I'm making some assumptions about the conditions; a mid size community with a straight level road within 5-20 feet of the tracks at grade.
This is a difficult question, I would look at when and where you are representing, as well as what prototypical practices may have been. I'll try to separate the factors into categories to keep things organized(ish).
There are several factors that will be very important; in no particular order they are:
1. Proximity.
2. Density.
2a. Cross Streets.
3. Geometry, Switches, and sidings.
4. Time and location.
I'll try to answer as generically as possible, perhaps others can add specifics.
1. How close the track and road are is significant, not the property lines but the edges of the travel areas. How much separation is there (vertical distance is more important than horizontal)? The closer the rails get to the pavement the slower it is going to be. Is there a delineation such as a guard rail (crash barrier) or fence? If there is a barrier the speed will come back up, the strength of the barrier will also be a factor. the barrier will be there for one or both of the following reasons, trespass or accident control; essentially intentional and unintentional incursion. A fence to prevent people from cutting across the tracks and mark the Right Of Way (ROW) as being separate from the street would increase the speed lack would slow it. This could be as simple as a single rail fence or as complex as a security fence complete with anti-scaling devices, increased space with vegetation will also increase speed. A guard rail of crash barrier would also increase the speed as that would prevent accidental incursion by vehicles. Both would be better than either alone. The answers to the next few points can determine which one of these is more important.
2. Density can be broken into roughly four categories; density of population, density of kind of population, density of rail traffic, density of road traffic. By density of population what I mean is how big a town/city is this "Main st." in? The extremes on either end will raise the speed, as a large city will necessitate an increase in speed to reduce congestion and a small rural community will not have enough people to justify slowing down. Additionally how dense is the neighborhood you are going through; is it crammed full of multi story building sitting right next to each other with an alley ever few buildings, or is it single or double story buildings separated by yards or parking lots? This leads to the density of the kind of population, a commercial or residential area will have higher traffic and that will drive down the rail speed. Density of rail traffic means the more trains the higher the speed, longer trains can also bring up the speed. Density of road traffic means that the more traffic on the road the lower the rail speed. Pedestrian traffic is generally more of a factor than vehicular traffic.
2a. I put cross streets here as they are kind of a bridge between topics 2 and 3. The frequency and type of at grade crossings will affect the speed, while grade separation will increase speeds, dense at grade un-signaled crossings will reduce the speed. How many of the local side streets that are continued over the ROW will greatly affect the speed.
3. Geometry means the grade and curvature of the track, coupled with the number of switches and tracks present, and with track condition; this is normally where railroads start when setting speed limits. Curves will reduce speeds as the radius gets smaller, grades will affect speed as well as steeper longer descents can mean lower speeds. The more switches there are the slower the speed will be as switches increase the probability of derailment. This means that a single line full of switches for local industries will be slower than a clear multi track mainline.
4. Time and location does become important as different areas have different rules and traditions about the proximity of rail traffic to the population, different eras will change this as well. Look for similar installations to when and where you are trying to represent, pictures of these may show what would be in place for physical characteristics while locations would allow you to find any special instructions or changes in the speed limit by the particular railroad operating there.
