Cindy Hyde-Smith sworn in by Mike Pence

Cindy Hyde-Smith was sworn in Monday as the junior senator from Mississippi, making her the first female member of Congress from the Southern state.

Vice President Mike Pence delivered the oath of office to Mrs. Hyde-Smith, the interim replacement for Sen. Thad Cochran, who stepped down on April 1 because of health issues.

Toting a large white Bible for the ceremony, Mrs. Hyde-Smith, a Republican who had been serving as Mississippi’s commissioner of agriculture and commerce, was accompanied by her fellow senator from Mississippi, Sen. Roger F. Wicker, and her husband, Michael Smith.

“What a day,” she told Mr. Pence.

Her appointment keeps the Republicans’ 51-49 Senate majority in place but does stretch the record number of women serving in the Senate at once to 23.

“It’s just very humbling, very humbling — very honored,” she said as she whisked through the Capitol hallways. “Just getting started, ready to go to work.”

Her historic ascension to the Senate coincided with another first for the august chamber. Sen. Tammy Duckworth, Illinois Democrat, became the first sitting senator to give birth. She had a second daughter, Maile Pearl Bowlsbey.

Mrs. Hyde-Smith cast her first vote late Monday in favor of the nomination of Claria Horn Boom to be a U.S. district judge for the Eastern and Western districts of Kentucky.

Senate Majority Leader Mitch McConnell, Kentucky Republican, welcomed the latest member of his caucus, saying she will advance “this Congress’ record of accomplishment for the American people.”

Senate Minority Leader Charles E. Schumer, New York Democrat, also welcomed her.

Mrs. Hyde-Smith, 58, served as a Democrat in the state Senate before becoming a Republican in 2010 and winning two statewide races. She is taking the seat Mr. Cochran held for 45 years while rising to chairman of the powerful Senate Appropriations Committee.

The newly minted senator recently told The Meridian Star editorial board in Mississippi that she will focus on upholding “conservative values” and helping President Trump enact his agenda.

She also wants to boost her state’s economy, defend the Second Amendment, strengthen the military and chip away at government regulations, the newspaper reported.

Mrs. Hyde-Smith will have little time to settle in before she faces a series of challengers in a November election, including former Democratic Rep. Mike Espy and Republican state Sen. Chris McDaniel, who nearly won the seat in a bitter 2014 race.

The winner of that race will serve out the remainder of Mr. Cochran’s term, through January 2021.

A poll released by the Espy campaign last week showed him in the lead against either possible Republican candidate, who split votes, according to the Clarion Ledger.

The poll did not include Democrat Jason Shelton, mayor of Tupelo, who announced he intends to run for the Democratic nomination, which could draw votes away from Mr. Espy.

Rather than facing party primaries, the candidates will all compete on Election Day in November. If no one secures a majority of the vote, the top two finishers will vie in a runoff later in the month.