Who will replace Petraeus atop the CIA?

So who might be the replacement for David Petraeus? The rumor mill was in full swing Friday after the CIA director stepped down, saying he had an extramarital affair.

One person being discussed is Michael Morell, the now acting CIA director, who could be named to the position permanently.

President Barack Obama thinks highly of Morell, several U.S. officials told CNN's Chief White House Correspondent Jessica Yellin.

In his statement on the resignation of Petraeus, Obama expressed the "utmost confidence" in Morell continuing the work of the CIA.

Senate Intelligence Committee chair Dianne Feinstein also expressed support of Morell, saying "the agency is in very good hands until the president selects a replacement."

But at the top could very well be John Brennan, the president's powerful adviser for counterterrorism and homeland security.

Brennan has shaped the White House's strategy to aggressively pursue suspected terrorists, dramatically escalating the use of armed unmanned aircraft to kill them in the ungoverned territories of Pakistan and in Yemen. Small teams of special operations forces have been deployed to critical locations. And Brennan is said to have the complete trust of Obama.

What could hold him back is exactly why he pulled out of contention for the CIA director's position in 2008 when the newly elected Obama was putting together his administration. Liberal bloggers accused Brennan of supporting outgoing President George W. Bush's policy of harsh interrogation techniques.

Brennan withdrew his name from consideration in an angry letter to Obama.

"It has been immaterial to the critics that I have a been strong opponent of many of the policies of the Bush administration such as preemptive war in Iraq and coercive interrogation tactics," Brennan wrote.

In the end, Brennan wound up with a more powerful position in the administration.

More recently, the White House has been criticized by some Republicans for leaks of sensitive information on counterterrorism operations. Brennan has called the leaks devastating and vehemently denied administration officials were involved. But the controversy could make for a difficult confirmation hearing if he is nominated.

Although many people say they think Brennan is ready to leave the administration after four years in an extremely demanding job, some who are close to him believe he still has his eyes set on the CIA job.

Another possibility for the position: Jane Harman, the former congresswoman from California who is well-respected within intelligence circles.

While in Congress she served as the ranking member of the House Intelligence Committee, chaired the Homeland Security Subcommittee on Counterterrorism and Intelligence, and was a member of the Armed Services Committee. Although a social liberal, Harman was seen as a moderate on intelligence and defense issues. She voted for the war in Iraq and initially supported the controversial domestic surveillance program run by the Bush administration.

Harman left Congress to run the Woodrow Wilson International Center, but stillspeaks out often on intelligence and foreign policy issues.

Undersecretary of Defense for Intelligence Michael Vickers is another name being bandied about.

Vickers has spent most of his career in the military and the intelligence community. Early on, he served as an Army Green Beret and a CIA operations officer. He helped arm the mujahedeen in Afghanistan in their battle against the Soviet occupation in the 1980s. His role in that operation was depicted in the 2007 movie, "Charlie Wilson's War," based on a book written by George Crile. Vickers later was named Assistant Secretary of Defense for Special Operations.

In his current position, Vickers simultaneously serves as the director of defense intelligence in the Office of the Director of National Intelligence, and as the primary military liaison to the CIA. His extensive experience working with the intelligence community and special operations forces is particularly noteworthy at a time when both services are a key component of counterterrorism strategy.

Another potential candidate is Mike Leiter, the former director of the National Counterterrorism Center (NCTC). The Harvard Law School graduate spent six years in the Navy flying missions over Bosnia and Iraq. After a stint as a federal prosecutor, Leiter served as the assistant director of a presidential commission on weapons of mass destruction, and was later named deputy chief of staff for the director of National Intelligence.

President George W. Bush appointed him director of NCTC, a position he continued in during the Obama administration. As director, Leiter oversaw more than 1,000 analysts charged with sifting through information gathered throughout the intelligence community in an effort to prevent terrorist attacks.

Although well-respected with strong support on Capitol Hill, Leiter did face criticism for the NCTC's failure to connect the dots when a Nigerian man tried unsuccessfully to blow up a Detroit bound airliner on Christmas Day in 2009.