It is not the first time my Icinga suddenlty, stops showing the grafana graphs.

Icinga when cannot show Grafana graphs

Problem

If trying to log to Grafana, admin password is not accepted. I am pretty sure i have not changed the password but I am unable to login to Grafana.

Invalid username or password when login to Grafana


And even worse, given that I still have not configured Grafana SMTP section / admin email address (set as admin@localhost), cannot ask Grafana to send me the reset password email.

Failed to send email when trying to reset password in Grafana

Update password in database

The only way I have to reset the password is to change it directly in the database.
Supposing we are using sqlite, this is the way to reset the password to “admin”.

# sqlite3 /var/lib/grafana/grafana.db

sqlite> update user set password = '59acf18b94d7eb0694c61e60ce44c110c7a683ac6a8f09580d626f90f4a242000746579358d77dd9e570e83fa24faa88a8a6', salt = 'F3FAxVm33R' where login = 'admin';

sqlite> exit

Then we can login again to Grafana, rechange the password to a non default one and check Icinga again is showing the graphs.

Graphana graph in Icinga

Configure SMTP

To avoid having to connect to the database directly again to solve a problem like this. Let’s configure the Grafana SMTP.
As a prerrequisite, a local MTA like Exim is supposed to be running locally (for example, the one that is using Icinga).

Configure grafana.ini

In the Grafana configuration file, edit /etc/grafana/grafana.ini, to enable SMTP:

############################ SMTP / Emailing ##########################
[smtp]
enabled = true
host = localhost:25
;user =
# If the password contains # or ; you have to wrap it with trippel quotes. Ex """#password;"""
;password =
;cert_file =
;key_file =
;skip_verify = false
from_address = grafanaserver@mydomain.com
;from_name = Grafana
# EHLO identity in SMTP dialog (defaults to instance_name)
;ehlo_identity = dashboard.example.com

And restart it:

# service grafana-server restart

Configure admin email

Inside Grafana console, edit admin profile and set a valid email (instead of the default admin@localhost)

Grafana Admin profile name, email and username

Test reset password email

Now it is time to check reset password email is working correctly

Email sent when resetting password in Grafana
Reset password link for Grafana received by email

No more need of changing the password in the database! 🙂