My issue with assessing trail difficulty is that I suspect my experience is unusual. I may simply not have seen very much of the 'easy trail' of which you spoke!
I hike nowadays mostly in upstate New York. A few decades ago, I hiked mostly New Hampshire.When I was in high school, I hiked Harriman a lot because I was a NYC kid. To a city kid, Harriman looked tough. Then I went to school in New Hampshire, and learnt otherwise! To my eyes now, Harriman mostly looks 'dead easy' except, say, for the scrambles on the Suffern-Bear Mountain trail, which are interesting.
I'd assess the hike I described as being comparable terrain, say, with the AT over Moosilauke forty years ago before they built the steps up Beaver Brook, or maybe the stretch from Pinkham Notch to US 2. The 'add a factor for rock scrambling' was for one pitch that looks like
https://www.flickr.com/photos/mounta...ons/3184443804 at the bottom and includes a pretty sketchy slab traverse
https://www.flickr.com/photos/mounta...ons/3184443344 that tops out with a waist-high mantel move
https://www.flickr.com/photos/mounta...ons/3184442762. No margin for error, if you wipe out you're going down that near-vertical slab from treetop height. I call it 'class 4' because in bad weather I want a rope.
There are lots of scrambles - the south side of Peekamoose is all 'hike 100 yards, of level trail, scramble a ledge, hike another 100 yards of level trail, scramble a ledge, lather, rinse, repeat" - but that one was the crux on that particular day,
Is that 'very difficult'? It's not too unusual for a Catskill or Adirondack trail. Saddleback and Gothics in the Adirondacks are harder, as are Twin and Sugarloaf in the Catskills.
I've not tried any long section on the AT. On the 137-mile Northville-Placid, I started out with a plan for 8-12 mile days and found out that 12-16 was more comfortable. But that trail is on relatively easy terrain; the chief challenges are the mud and the remoteness (there are two 40-mile sections that don't even cross logging roads).
On my personal scale, Harriman, southern VT or central MA on the AT are all 'pretty easy.' I've always figured that they were the exception rather than the rule.
Also, I do stop a lot when I'm just getting back into things - which seems to be all the time, lately, drat it. Until I've been hiking for a week or two, I can't sustain hiking all day long from sunup to sundown, and I'll get hurt if I try. So I lallygag - take pictures, write, watch wildlife, update trail maps, socialize - for a total of a few hours during the day while my body is getting used to hiking again. By the time I am used to hiking, it's time to go back to work. I need those stops.
I suppose I go briskly enough when I'm moving. Certainly I'd do the occasional 15-in-an-afternoon on the Erie Canal towpath; that'd take about five hours with a day pack. The point of 'stop and smell the roses' isn't about walking pace, it's about it being ok to be interested in other things than hiking - even when on a hike. Even roses, if there are any to be smelt.