Thank you for your qurestion!
Lesson 6.5 is about comparative endings. In most cases you just need to add er to the adjective, such as “Der Mann ist kleiner.” - “The man is shorter.”
When comparatives are placed before a noun however, you need to add the appropriate adjective endings, such as “der kleinere Mann.” - “The shorter man”.
“der kleinere Mann” is nominative masculine singular which means you need to add an e to the comparative form er. If it wasn't comparative, it would be “der kleine Mann” - "the short man".
Explanations on these endings can be found in both lesson 5.8 ‘Adjectives in the Nominative and Accusative’ and lesson 10.8 ‘Adjective Endings’.
The next sentence you are referring to is:
“Haben Sie längere Kleider?” - “Do you have (any) longer dresses?”
Here, there is no article in front of the adjective which means the adjective itself has to indicate the case in place of the article. If there was an article, the sentence would change into:
“Haben Sie die längeren Kleider?” - “Do you have the longer dresses?” (accusative, neuter, plural)
But since there is no article, the adjective needs to take the ending of the definite article so it becomes: “Haben Sie längere Kleider?” - “Do you have (any) longer dresses?”
I hope this helps!