Is CHARSET important for tables with INT only?

My server default CHARSET is UTF-8 (set in my.cnf). I create tables like

CREATE TABLE t1 ( ID int(11) unsigned NOT NULL AUTO_INCREMENT, Title varchar(255), PRIMARY KEY(ID) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_general_ci KEY_BLOCK_SIZE=2 

but some tables have only int columns. Is it beneficial to use latin charset for them? Or is it better to have all tables with UTF-8 charset for consistency?

CREATE TABLE t2 ( ID int(11) unsigned NOT NULL, TagID int(11) unsigned NOT NULL, PRIMARY KEY(ID,TagID) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_general_ci KEY_BLOCK_SIZE=1 

vs.

CREATE TABLE t2 ( ID int(11) unsigned NOT NULL, TagID int(11) unsigned NOT NULL, PRIMARY KEY(ID,TagID) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE latin1_swedish_ci KEY_BLOCK_SIZE=1