thank you for your question!
Yes, nett is describing the dinner, but instead of just referring to just the noun (e.g. "nettes Abendessen"), it is referring to the infinitive clause "...zum Abendessen zu gehen"
Let me give you some other examples:
"Wir haben beschlossen, (heute) schön essen zu gehen." -"We have decided to go out for a nice meal (today)."
"Wir haben beschlossen, (heute) laut beim Konzert zu singen." - "We have decided to sing loudly at the concert (today)."
"Er hat beschlossen, (heute) nett zum Kollegen zu sein." -"He has decided to be nice to the colleague (today)."
I hope this was helpful!