An ore refining complex in St. James Parish has had so many serious worker health and safety violations in the past year that federal regulators say they will pull its workers off the job if another serious violation is found.