US says it will return nearly $650,000 in fines to fishermen