Curious, were the double-decker shinkansens not being used to capacity, or are the new single-deck trains going to be twice as long?
From what I have read it is because they want to speed up the Tohoku Shinkansen and the Joetsu in the future. And the E4 runs "only" 240 km/h max they have decided to phase out this trains. They are probably also on the end of their service life. From the video's that I saw they where not used to full capacity anymore. Its sad to see them go. Luckily we got them in Trainz.
The replacement of E4 Shinkansen trains will be the single-deck E7 Series, wich is already in service. The issue with the double-decker shinkansen trains was indeed their double deck: due to the larger passenger capacity, while keeping the same two doors per side as single-level shinkansens, the dwell time for the E4 Series is far larger than other trains, meaning that the train sits at the station for longer, and thus the frequency of services on the line is limited.
By removing the E4 Series, JR East will be able to increase the frequency of the lines and thus will be able to "squeeze in" a couple more trains in the tightly packed schedules. Therefore, counter-intuitively, removing the highest-capacity shinkansen trains will be actually beneficial to the line's overall capacity.
Double-decker shinkansen trains are an oddity, a by-product of the Tohoku and the Joets Shinkansen lines' particular planning history, we could say. The "original" Tokaido Shinkansen was built to relieve congestion on the saturated Tokaido Main Line and scheduling was competitive with airline travel, therefore ridership (and thus revenue) was almost assured from the start. A similar thing happened in 1975 with the San'yo Shinkansen: while the San'yo Main Line wasn't as congested as the Tokaido Main Line and airlines could still have a sizeable market share, the San'yo Shinkansen ran trough a densely populated and well-developed area, ensuring even here a sizeable ridership, altough not as much as the Tokaido Shinkansen.
The matter with the Tohoku and Joetsu Shinkansen lines was entirely different: the regions north of Tokyo were (and still partly are to this day) predominantly rural and underdeveloped (compared to the rest of Honshu). For the first time the matter of planning a Shinkansen line wasn't left entirely to the National Railways, but was taken over mostly by political entities, with the Shinkansen no longer being used as a relief for conventional lines, but rather as a political tool (some might say a "pork barrel"), officially to "establish closer ties with Tokyo and to promote regional development".
In the end, the cost of building new Shinkansen lines for a low ridership was one of the main contributing factors to JNR's debt collapse and subsequent privatization in 1987. Upon it's formation, JR East inherited the Tohoku and Joetsu shinkansen lines as it's only high-speed lines, with both having ridership far lower than the minimium needed to break even.
Thus, in an attempt to gain ridership, for the first time, the Shinkansen began to be "marketed" not only to long-distance travellers, but also to commuters as well. The E1 and E4 double-decker shinkansens were a result of this "commuter shinkansen" concept, as they were intended to be used for short services (Ueno to Takasaki or Utsunomiya - services later extended further "outwards" for practical reasons) dedicated for commuters heading into Tokyo, hence the reason for their two decks and lower speed than other shinkansen trains: maximized seated capacity (to contain running costs) while running frequent stopping services.
This was at a time where the two lines were still relatively underutilized, and thus the impact of the longer station dwell times was trascurable. Only from the late 2000s onwards double-decker shinkansen began to be a burden: with the urbanization of rural areas and the extension of the Tohoku Shinkansen northwards, ridership increased dramatically. Add to that the opening of the Nagano Shinkansen (currently the Hokuriku Shinkansen) wich branched off the Joetsu Shinkansen at Takasaki and the original "commuter area" section intended to be served by double-decker shinkansens quickly became saturated with traffic: between Ageo and Tokyo three different Shinkansen lines had to use the same pair of tracks, thus evry second counts.
Furthemore, with profits secured from the increasing ridership, JR East no longer needed the "commuter shinkansen" concept: now squeezing more trains in the timetables took priority, and to do that, getting a train in-and-out of a station as quick as possible is key, something that double-decker shinkansen were unable to do, and thus, led to their retirement.
This is roughly the same problem found on high-traffic conventional lines, such as the Yamanote Line: why don't they use double-decker cars instead of single-level ones? Yes, they're single-level, but they have four doors per side. If the Yamanote Line was to use double-decker cars with two doors only, it's capacity would be reduced to a third of what it is now.
Something related to this happened in few years ago in France: on Line A of the Paris RER, the world's busiest railway line outside Japan, the introduction of the new MI09 double-decker trains actually hampered the line's capacity, wich was reduced by a couple of trains-per-hour, compared to when the line was operated with the single-level MI84 stock (one of europe's finest electric multiple units) wich had four doors per side. This was even depsite the fact that MI09s are an exceptional case within double-decker trains as they're fitted with thee doors per side (like their MI2N predecessors) instead of the far more commonplace two.