In Acts 13:22, Paul speaks in the synagogue and explains “And when he [God] had removed him [King Saul], he raised up David to be their king, of whom he testified and said, 'I have found in David the son of Jesse a man after my heart, who will do all my will.'” So the contrast between Saul and David seems to be that Saul did not do God's will, but David sincerely desired to, and in fact would do God's will. This phrase “a man after my heart” is a reference to 1 Samuel 13:14. In that 1 Samuel 13 passage, Saul had been instructed to wait in a certain place for Samuel to come and make a sacrifice to God before going into battle. But while Samuel seemed to be taking too long, Saul got impatient and offered the sacrifice himself. When Samuel showed up, he told Saul these words; that as a result of this sin, “your kingdom shall not continue. The Lord has sought out a man after his own heart, and the Lord has commanded him to be prince over his people, because you have not kept what the Lord commanded you.” (1 Samuel 13:14). In offering the sacrifice himself, Saul demonstrated that he had no respect for the divine order for the nation. The king was to be under God, and prophets like Samuel spoke the word of God to the kings. Samuel had essentially put himself above God. If we think this seems like a fairly minor offense for which the punishment is too harsh, we need to understand that this was Israel’s first king. God had to demonstrate the right order of things before all Israel. If the king would undermine God's order for the nation, then nobody should take it seriously. Saul's disregard for God's law before the people showed that he was not a suitable king for Israel.
Now David, on the other hand, respected and upheld God's law. David applied God's law to all that he did. When he fought Goliath, the giant, he was confident because he knew that it was God's will for Israel to conquer their enemies. When Saul was seeking to kill him, David would not, himself, take Saul's life because (he said) “The Lord rewards every man for his righteousness and his faithfulness, for the Lord gave you into my hand today, and I would not put out my hand against the Lord's anointed.” (1 Samuel 26:23). David was a righteous example for the people, and he did God's will in obeying His law and conquering the enemies of Israel.
Now the problem we typically have with David being described as “a man after God's own heart” is that, in a most serious manner, he didn't obey God's law. He committed adultery with Bathsheba and murdered her husband. How can this man be regarded as a man after God's own heart? And why wasn't the kingdom taken from David for this sin, whilst it was taken from Saul for his sin in making a sacrifice to God when he had been instructed to wait for Samuel? Well, let's consider what it means to be “after God's own heart”. That word “after” seems, to me, to be in that same sense as when we talk about “chasing after something.” It's the case that David, and we also, do not have the heart of God, but are “chasing after” it. When we keep this in mind, we can see how David and Saul differed. David wanted always to be God-like (that is, to have the heart of God). When David sinned he recognized his sin and, turning his mind back to “the chase for God's heart”, repented. He realized that what he had done was not at all God-like and he turned away from it, back towards a Godly attitude. His very attitude towards his own sin was God-like. Having committed adultery with Bathsheba, he writes a Psalm of repentance to God; Psalm 51. He writes, there, for example:
Hide your face from my sins, and blot out all my iniquities. Create in me a clean heart, O God, and renew a right spirit within me. (Psalm 51:9-10)
Here, David is clearly recognizing that his heart has not been the heart of God, and he is praying for God to restore in him that “clean heart” and a “right spirit”.
Back in 1 Samuel 13, when Saul had committed the sin of the sacrifice, Samuel confronts him. Saul, rather than having this repentant heart, makes excuses for himself saying, “When I saw that the people were scattering from me, and that you did not come within the days appointed, and that the Philistines had mustered at Michmash, ... I forced myself, and offered the burnt offering.” (1 Samuel 13:11-12). He is not repentant but defensive. Now Saul's sin was disregard for God's “cultic laws”. David, too, erred in the cultic laws of God. When moving the Ark of the Covenant to Jerusalem, he had loaded the Ark onto a cart. But the law of God said that the Ark should be carried, by hand, by the priests. Again, why wasn't the kingdom taken from David, even for a “similar” sin? It is because, in as much as David was a bad example to the people, he quickly made himself a good example of repentance. He said “Because you [the priests] did not carry it the first time, the LORD our God broke out against us, because we did not seek him according to the rule.” (1 Chronicles 15:13). He recognized his error, and in 1 Chronicles 15 we read about David's second preparation to bring the Ark in which he has the priesthood consecrate themselves for the task.
Whenever David missed the mark of God-likeness, his sins did not go unpunished. For the sin with Bathsheba, for example, God took the life of his son which he had with Bathsheba. In fact, for just about every sin of David we can find a consequence. But the reason Saul lost the kingdom but David didn't is because Saul was never a godly person, seeking God's will and “chasing after the heart of God”. He would never be a good example for the people, or a good leader. David, on the other hand, would continually return to God in repentance over his sins, precisely because he was continually chasing after God's own heart. He remains, to this day, one of the greatest exemplars of faith in the Bible, and was the “benchmark” for all of the kings of Judah throughout the historical-narrative books of the Bible.
The sacrifices of God are a broken spirit; a broken and contrite heart, O God, you will not despise. (Psalm 51:17)