Jul 28 14:22:49.085862 ip-10-0-131-88 systemd[1]: Starting Kubernetes Kubelet... Jul 28 14:22:49.476435 ip-10-0-131-88 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 14:22:49.476435 ip-10-0-131-88 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 14:22:49.476435 ip-10-0-131-88 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 14:22:49.476435 ip-10-0-131-88 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 14:22:49.476435 ip-10-0-131-88 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 14:22:49.478569 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.478471 2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 14:22:49.485653 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485631 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 14:22:49.485653 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485649 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 14:22:49.485653 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485659 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485662 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485667 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485670 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485673 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485675 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485678 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485681 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485683 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485686 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485689 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485691 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485694 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485697 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485699 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485702 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485705 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485708 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485710 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485713 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 14:22:49.485800 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485715 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485718 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485720 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485724 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485726 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485729 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485732 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485735 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485738 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485744 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485747 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485749 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485752 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485755 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485758 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485761 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485763 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485766 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485769 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 14:22:49.486279 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485771 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485774 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485777 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485780 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485782 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485785 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485787 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485790 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485792 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485795 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485798 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485801 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485804 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485806 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485809 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485812 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485815 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485819 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485823 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 14:22:49.486768 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485826 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485829 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485832 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485834 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485837 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485840 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485842 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485846 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485850 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485852 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485855 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485858 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485860 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485863 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485866 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485868 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485872 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485876 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485879 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 14:22:49.487225 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485882 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485886 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485890 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485893 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485897 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485900 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485903 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485905 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485908 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485911 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485913 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485916 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485918 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485921 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485924 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485926 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485929 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485932 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485934 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.485937 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 14:22:49.487695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486648 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486655 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486659 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486661 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486664 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486667 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486670 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486673 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486675 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486678 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486680 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486683 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486686 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486688 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486691 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486694 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486697 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486701 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486703 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486706 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 14:22:49.488174 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486709 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486711 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486715 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486719 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486722 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486726 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486729 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486732 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486734 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486737 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486740 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486743 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486746 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486749 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486751 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486754 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486757 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486760 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486763 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 14:22:49.488676 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486767 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486770 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486773 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486776 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486779 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486782 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486784 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486787 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486790 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486792 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486794 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486798 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486801 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486803 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486806 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486809 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486811 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486814 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486817 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486820 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 14:22:49.489145 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486823 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486825 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486828 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486830 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486833 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486836 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486838 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486841 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486844 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486847 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486852 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486856 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486859 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486862 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486864 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486867 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486869 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486872 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486874 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 14:22:49.489655 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486877 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486879 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486882 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486884 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486887 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486890 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486893 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486895 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486898 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486900 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486903 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486906 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486908 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486911 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486914 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486916 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486919 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486922 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486925 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486927 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 14:22:49.490110 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.486930 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487290 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487295 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487298 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487300 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487303 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487306 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487309 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487311 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487314 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487318 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487322 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487325 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487330 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487333 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487336 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487339 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487342 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487345 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487348 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 14:22:49.490605 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487351 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487354 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487356 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487359 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487362 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487365 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487367 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487370 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487373 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487375 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487378 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487380 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487383 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487385 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487388 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487390 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487393 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487395 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487398 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487400 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 14:22:49.491111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487403 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487406 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487408 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487411 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487414 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487418 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487422 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487424 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487427 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487429 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487433 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487436 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487438 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487441 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487444 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487446 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487449 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487451 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487453 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 14:22:49.491600 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487456 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487458 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487461 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487463 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487466 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487468 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487471 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487473 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487476 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487478 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487481 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487483 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487486 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487488 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487491 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487494 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487496 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487498 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487502 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487505 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 14:22:49.492059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487507 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487510 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487513 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487516 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487519 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487522 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487524 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487526 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487532 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487535 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487538 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487541 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487544 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487559 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487562 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487565 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487568 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487570 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487573 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 14:22:49.492571 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487576 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.487578 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487646 2586 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487653 2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487665 2586 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487672 2586 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487679 2586 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487684 2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487689 2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487696 2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487699 2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487704 2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487707 2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487711 2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487713 2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487716 2586 flags.go:64] FLAG: --cgroup-root="" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487719 2586 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487723 2586 flags.go:64] FLAG: --client-ca-file="" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487726 2586 flags.go:64] FLAG: --cloud-provider="external" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487729 2586 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487733 2586 flags.go:64] FLAG: --cluster-domain="" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487736 2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487739 2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487742 2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 14:22:49.493059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487746 2586 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487750 2586 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487752 2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487756 2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487759 2586 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487762 2586 flags.go:64] FLAG: --contention-profiling="false" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487765 2586 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487767 2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487771 2586 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487774 2586 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487778 2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487781 2586 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487784 2586 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487787 2586 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487790 2586 flags.go:64] FLAG: --enable-server="true" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487793 2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487798 2586 flags.go:64] FLAG: --event-burst="100" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487800 2586 flags.go:64] FLAG: --event-qps="50" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487803 2586 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487807 2586 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487809 2586 flags.go:64] FLAG: --eviction-hard="" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487814 2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487816 2586 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487820 2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487823 2586 flags.go:64] FLAG: --eviction-soft="" Jul 28 14:22:49.493648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487826 2586 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487829 2586 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487832 2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487836 2586 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487838 2586 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487841 2586 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487844 2586 flags.go:64] FLAG: --feature-gates="" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487850 2586 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487853 2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487856 2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487859 2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487862 2586 flags.go:64] FLAG: --healthz-port="10248" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487865 2586 flags.go:64] FLAG: --help="false" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487868 2586 flags.go:64] FLAG: --hostname-override="ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487871 2586 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487874 2586 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487876 2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487880 2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487883 2586 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487886 2586 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487888 2586 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487891 2586 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487894 2586 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 14:22:49.494520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487897 2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487900 2586 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487903 2586 flags.go:64] FLAG: --kube-reserved="" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487906 2586 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487908 2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487911 2586 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487914 2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487917 2586 flags.go:64] FLAG: --lock-file="" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487920 2586 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487923 2586 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487926 2586 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487931 2586 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487937 2586 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487940 2586 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487943 2586 flags.go:64] FLAG: --logging-format="text" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487946 2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487949 2586 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487952 2586 flags.go:64] FLAG: --manifest-url="" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487955 2586 flags.go:64] FLAG: --manifest-url-header="" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487959 2586 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487962 2586 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487966 2586 flags.go:64] FLAG: --max-pods="110" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487969 2586 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487972 2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487975 2586 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 14:22:49.495564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487977 2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487980 2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487983 2586 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487986 2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487994 2586 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487996 2586 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.487999 2586 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488002 2586 flags.go:64] FLAG: --pod-cidr="" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488005 2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488010 2586 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488013 2586 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488015 2586 flags.go:64] FLAG: --pods-per-core="0" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488018 2586 flags.go:64] FLAG: --port="10250" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488021 2586 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488024 2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01f2490cd2f37e485" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488030 2586 flags.go:64] FLAG: --qos-reserved="" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488033 2586 flags.go:64] FLAG: --read-only-port="10255" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488036 2586 flags.go:64] FLAG: --register-node="true" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488039 2586 flags.go:64] FLAG: --register-with-taints="" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488042 2586 flags.go:64] FLAG: --registry-burst="10" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488046 2586 flags.go:64] FLAG: --registry-qps="5" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488049 2586 flags.go:64] FLAG: --reserved-cpus="" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488052 2586 flags.go:64] FLAG: --reserved-memory="" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488056 2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488059 2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 14:22:49.496641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488061 2586 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488064 2586 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488067 2586 flags.go:64] FLAG: --runonce="false" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488070 2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488073 2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488076 2586 flags.go:64] FLAG: --seccomp-default="false" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488078 2586 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488081 2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488084 2586 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488087 2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488090 2586 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488093 2586 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488096 2586 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488098 2586 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488101 2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488104 2586 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488107 2586 flags.go:64] FLAG: --system-cgroups="" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488110 2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488115 2586 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488118 2586 flags.go:64] FLAG: --tls-cert-file="" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488121 2586 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488124 2586 flags.go:64] FLAG: --tls-min-version="" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488128 2586 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488131 2586 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488134 2586 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 14:22:49.497271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488137 2586 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488140 2586 flags.go:64] FLAG: --v="2" Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488144 2586 flags.go:64] FLAG: --version="false" Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488148 2586 flags.go:64] FLAG: --vmodule="" Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488152 2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.488155 2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488267 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488270 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488291 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488295 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488298 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488301 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488304 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488307 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488310 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488313 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488317 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488319 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488322 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488324 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488327 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 14:22:49.498260 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488330 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488332 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488335 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488339 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488341 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488344 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488347 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488349 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488353 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488356 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488358 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488361 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488363 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488366 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488369 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488372 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488374 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488377 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488379 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488382 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 14:22:49.499175 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488385 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488387 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488390 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488392 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488395 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488398 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488400 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488403 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488405 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488408 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488412 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488416 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488419 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488422 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488425 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488429 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488432 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488434 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488437 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 14:22:49.499811 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488440 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488443 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488447 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488451 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488453 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488456 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488459 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488462 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488464 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488468 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488470 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488473 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488477 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488480 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488484 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488487 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488489 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488492 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488495 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 14:22:49.500618 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488498 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488500 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488503 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488505 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488508 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488511 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488513 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488516 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488519 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488523 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488525 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488528 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488530 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488533 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488537 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488539 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488542 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488545 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488559 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488562 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 14:22:49.501270 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488564 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488567 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488570 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488573 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488575 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.488578 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.489147 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.496614 2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.496635 2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.497422 2586 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.497435 2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497533 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497542 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497564 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 14:22:49.501912 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497571 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497579 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497585 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497590 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497595 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497600 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497606 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497610 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497615 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497620 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497624 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497628 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497633 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497637 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497641 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497646 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497651 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497655 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497659 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497663 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 14:22:49.502282 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497668 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497672 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497676 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497680 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497684 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497688 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497702 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497709 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497715 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497720 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497724 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497729 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497734 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497739 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497743 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497747 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497751 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497756 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497760 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 14:22:49.502794 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497764 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497768 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497772 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497776 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497781 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497785 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497789 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497794 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497798 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497802 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497807 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497811 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497815 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497819 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497824 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497828 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497832 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497837 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497840 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497845 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 14:22:49.503326 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497858 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497863 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497867 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497871 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497876 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497880 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497884 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497888 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497893 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497897 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497901 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497905 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497909 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497922 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497927 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497931 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497935 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497939 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497943 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497948 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 14:22:49.503945 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497953 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497957 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497961 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497965 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497970 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497974 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497979 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497983 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497987 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497991 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.497995 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498000 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498004 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498017 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498022 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498027 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 14:22:49.504428 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498031 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.498039 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498259 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498266 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498271 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498275 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498280 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498287 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498294 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498298 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498303 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498307 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498312 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498316 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 14:22:49.504845 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498320 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498324 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498328 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498333 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498337 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498341 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498345 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498349 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498354 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498358 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498362 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498366 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498370 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498374 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498379 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498384 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498396 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498401 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498406 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498410 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 14:22:49.505185 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498414 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498418 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498423 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498427 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498431 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498435 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498439 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498443 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498447 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498454 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498460 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498464 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498468 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498473 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498477 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498481 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498486 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498490 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498494 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 14:22:49.505718 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498498 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498503 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498507 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498511 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498515 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498519 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498523 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498528 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498532 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498536 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498564 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498570 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498574 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498578 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498583 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498587 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498591 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498596 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498600 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498603 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 14:22:49.506457 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498608 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498612 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498617 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498621 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498625 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498629 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498633 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498638 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498642 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498646 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498650 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498654 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498659 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498663 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498668 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498672 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498675 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498680 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498684 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498688 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 14:22:49.507308 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498692 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498697 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498701 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498714 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498718 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498722 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498726 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:49.498732 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.498740 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.498913 2586 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.501641 2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.503041 2586 server.go:1019] "Starting client certificate rotation" Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.503167 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 14:22:49.508101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.503204 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 14:22:49.508621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.506612 2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 14:22:49.510217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.510197 2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 14:22:49.522980 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.522962 2586 log.go:25] "Validated CRI v1 runtime API" Jul 28 14:22:49.525782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.525769 2586 log.go:25] "Validated CRI v1 image API" Jul 28 14:22:49.526529 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.526517 2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 14:22:49.528869 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.528847 2586 fs.go:135] Filesystem UUIDs: map[6a9a38d2-a7ab-4a8f-aa22-6df57133d6e0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d2f8dd91-8474-45e6-89b7-d478e05177e5:/dev/nvme0n1p4] Jul 28 14:22:49.528939 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.528869 2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 14:22:49.533260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.533243 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 14:22:49.533962 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.533854 2586 manager.go:216] Machine: {Timestamp:2026-07-28 14:22:49.532528535 +0000 UTC m=+0.333406082 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099610 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28f5cff675c9aece476287616dcf5a SystemUUID:ec28f5cf-f675-c9ae-ce47-6287616dcf5a BootID:8a0aea86-92c7-4ed0-9a63-d227c344943f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:bb:9e:aa:eb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:bb:9e:aa:eb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:70:44:de:ee:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 14:22:49.533962 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.533961 2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 14:22:49.534071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.534044 2586 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 14:22:49.537469 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.537435 2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 14:22:49.537610 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.537472 2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-88.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 14:22:49.538227 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.538217 2586 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 14:22:49.538260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.538228 2586 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 14:22:49.538260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.538236 2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 14:22:49.539093 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.539082 2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 14:22:49.540352 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.540343 2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 14:22:49.543000 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.542980 2586 state_mem.go:36] "Initialized new in-memory state store" Jul 28 14:22:49.543108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543099 2586 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 14:22:49.543136 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543128 2586 kubelet.go:486] "Attempting to sync node with API server" Jul 28 14:22:49.543164 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543137 2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 14:22:49.543164 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543156 2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 14:22:49.543234 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543166 2586 kubelet.go:398] "Adding apiserver pod source" Jul 28 14:22:49.543234 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543175 2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 14:22:49.543578 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543566 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 14:22:49.543617 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.543591 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 14:22:49.545886 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.545871 2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 14:22:49.547490 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.547476 2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 14:22:49.547610 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.547506 2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 14:22:49.548948 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548936 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548952 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548959 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548965 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548973 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548978 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548986 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548991 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 14:22:49.548997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.548997 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 14:22:49.549204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.549003 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 14:22:49.549204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.549016 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 14:22:49.549204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.549021 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 14:22:49.549792 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.549781 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 14:22:49.549792 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.549792 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 14:22:49.552366 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.552339 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 14:22:49.552516 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.552488 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-88.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 14:22:49.552733 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.552716 2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-88.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 14:22:49.554148 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.554131 2586 server.go:1268] "Started kubelet" Jul 28 14:22:49.554303 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.554243 2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 14:22:49.554303 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.554270 2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 14:22:49.554396 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.554327 2586 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 14:22:49.555009 ip-10-0-131-88 systemd[1]: Started Kubernetes Kubelet. Jul 28 14:22:49.555627 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.555452 2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 14:22:49.555627 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.555501 2586 server.go:310] "Adding debug handlers to kubelet server" Jul 28 14:22:49.560089 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.560057 2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 14:22:49.560839 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.559957 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-88.ec2.internal.18c67a1d880bd283 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-88.ec2.internal,UID:ip-10-0-131-88.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-88.ec2.internal,},FirstTimestamp:2026-07-28 14:22:49.554096771 +0000 UTC m=+0.354974318,LastTimestamp:2026-07-28 14:22:49.554096771 +0000 UTC m=+0.354974318,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-88.ec2.internal,}" Jul 28 14:22:49.562192 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562173 2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 14:22:49.562192 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562187 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 14:22:49.562316 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.562271 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:49.562798 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562781 2586 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 14:22:49.562896 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562801 2586 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 14:22:49.562953 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562916 2586 reconstruct.go:97] "Volume reconstruction finished" Jul 28 14:22:49.562953 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562924 2586 reconciler.go:29] "Reconciler: start to sync state" Jul 28 14:22:49.563052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563012 2586 factory.go:55] Registering systemd factory Jul 28 14:22:49.563052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563029 2586 factory.go:223] Registration of the systemd container factory successfully Jul 28 14:22:49.563190 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.562787 2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 14:22:49.563347 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563333 2586 factory.go:153] Registering CRI-O factory Jul 28 14:22:49.563347 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563348 2586 factory.go:223] Registration of the crio container factory successfully Jul 28 14:22:49.563466 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563335 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d9w2b" Jul 28 14:22:49.563466 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563385 2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 14:22:49.563466 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563428 2586 factory.go:103] Registering Raw factory Jul 28 14:22:49.563466 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.563443 2586 manager.go:1196] Started watching for new ooms in manager Jul 28 14:22:49.564182 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.564165 2586 manager.go:349] Starting recovery of all containers Jul 28 14:22:49.572311 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.572286 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d9w2b" Jul 28 14:22:49.572515 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.572487 2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-88.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 14:22:49.572776 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.572752 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 14:22:49.576404 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.576293 2586 manager.go:354] Recovery completed Jul 28 14:22:49.581933 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.581920 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.583533 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.583519 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.583633 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.583563 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.583633 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.583579 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.584090 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.584075 2586 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 14:22:49.584141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.584091 2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 14:22:49.584141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.584107 2586 state_mem.go:36] "Initialized new in-memory state store" Jul 28 14:22:49.584141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.584082 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.585627 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.585614 2586 policy_none.go:49] "None policy: Start" Jul 28 14:22:49.585688 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.585633 2586 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 14:22:49.585688 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.585647 2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 14:22:49.585755 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.585712 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.585755 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.585739 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.585755 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.585749 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.585854 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.585709 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-88.ec2.internal.18c67a1d89ccff5f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-88.ec2.internal,UID:ip-10-0-131-88.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-88.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-88.ec2.internal,},FirstTimestamp:2026-07-28 14:22:49.583533919 +0000 UTC m=+0.384411467,LastTimestamp:2026-07-28 14:22:49.583533919 +0000 UTC m=+0.384411467,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-88.ec2.internal,}" Jul 28 14:22:49.586676 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.586665 2586 policy_none.go:47] "Start" Jul 28 14:22:49.625081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625062 2586 manager.go:337] "Starting Device Plugin manager" Jul 28 14:22:49.625165 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.625095 2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 14:22:49.625165 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625106 2586 server.go:87] "Starting device plugin registration server" Jul 28 14:22:49.625589 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625577 2586 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 14:22:49.625648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625593 2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 14:22:49.625726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625691 2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 14:22:49.625786 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625770 2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 14:22:49.625786 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.625780 2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 14:22:49.626331 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.626303 2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 14:22:49.626401 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.626355 2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:49.683341 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.683303 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 14:22:49.684455 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.684438 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 14:22:49.684533 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.684480 2586 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 14:22:49.684533 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.684512 2586 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 14:22:49.684646 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.684571 2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 14:22:49.687173 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.687153 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 14:22:49.726106 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.726086 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.726959 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.726906 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.726959 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.726944 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.726959 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.726958 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.727154 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.726999 2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.746889 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.746869 2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.746972 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.746893 2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-88.ec2.internal\": node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:49.768659 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.768629 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:49.784665 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.784645 2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal"] Jul 28 14:22:49.784761 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.784717 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.787910 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.787893 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.787983 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.787923 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.787983 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.787933 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.789322 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.789308 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.789452 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.789436 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.789513 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.789475 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.790065 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.790053 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.790138 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.790084 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.790138 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.790057 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.790138 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.790124 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.790257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.790101 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.790257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.790143 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.791124 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.791106 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.791212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.791132 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 14:22:49.791898 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.791881 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientMemory" Jul 28 14:22:49.791991 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.791909 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 14:22:49.791991 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.791919 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeHasSufficientPID" Jul 28 14:22:49.818345 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.818326 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-88.ec2.internal\" not found" node="ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.823024 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.823008 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-88.ec2.internal\" not found" node="ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.864402 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.864377 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae61811891b0aefce4345602b66dee13-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal\" (UID: \"ae61811891b0aefce4345602b66dee13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.864493 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.864406 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae61811891b0aefce4345602b66dee13-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal\" (UID: \"ae61811891b0aefce4345602b66dee13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.864493 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.864428 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ff9d18249fb2db5835b1f8658874702-config\") pod \"kube-apiserver-proxy-ip-10-0-131-88.ec2.internal\" (UID: \"6ff9d18249fb2db5835b1f8658874702\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.868757 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.868736 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:49.965134 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.965107 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ff9d18249fb2db5835b1f8658874702-config\") pod \"kube-apiserver-proxy-ip-10-0-131-88.ec2.internal\" (UID: \"6ff9d18249fb2db5835b1f8658874702\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.965229 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.965141 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae61811891b0aefce4345602b66dee13-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal\" (UID: \"ae61811891b0aefce4345602b66dee13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.965229 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.965162 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae61811891b0aefce4345602b66dee13-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal\" (UID: \"ae61811891b0aefce4345602b66dee13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.965295 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.965230 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae61811891b0aefce4345602b66dee13-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal\" (UID: \"ae61811891b0aefce4345602b66dee13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.965295 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.965253 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ff9d18249fb2db5835b1f8658874702-config\") pod \"kube-apiserver-proxy-ip-10-0-131-88.ec2.internal\" (UID: \"6ff9d18249fb2db5835b1f8658874702\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.965295 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:49.965228 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae61811891b0aefce4345602b66dee13-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal\" (UID: \"ae61811891b0aefce4345602b66dee13\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:49.969329 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:49.969314 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.069872 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.069834 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.123002 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.122963 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:50.127166 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.126850 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" Jul 28 14:22:50.170936 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.170903 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.271510 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.271472 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.372660 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.372584 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.473564 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.473526 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.502815 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.502785 2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 14:22:50.503411 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.502940 2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 14:22:50.574421 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.574392 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.576575 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.576537 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 14:17:50 +0000 UTC" deadline="2028-01-25 16:12:07.534920323 +0000 UTC" Jul 28 14:22:50.576575 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.576572 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13105h49m16.958351682s" Jul 28 14:22:50.587099 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.587076 2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 14:22:50.607217 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:50.607189 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff9d18249fb2db5835b1f8658874702.slice/crio-1e4e7ea8f7c50fe6b003069c6c000cd5f8496ce8091ea49f50b685a97fc9ffe1 WatchSource:0}: Error finding container 1e4e7ea8f7c50fe6b003069c6c000cd5f8496ce8091ea49f50b685a97fc9ffe1: Status 404 returned error can't find the container with id 1e4e7ea8f7c50fe6b003069c6c000cd5f8496ce8091ea49f50b685a97fc9ffe1 Jul 28 14:22:50.612936 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.612923 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 14:22:50.675183 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.675113 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.678695 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:50.678655 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae61811891b0aefce4345602b66dee13.slice/crio-75620aa0eb9053783ed62e72e180e26fae7f52168e0ad2a32e04317816442665 WatchSource:0}: Error finding container 75620aa0eb9053783ed62e72e180e26fae7f52168e0ad2a32e04317816442665: Status 404 returned error can't find the container with id 75620aa0eb9053783ed62e72e180e26fae7f52168e0ad2a32e04317816442665 Jul 28 14:22:50.687856 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.687812 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" event={"ID":"ae61811891b0aefce4345602b66dee13","Type":"ContainerStarted","Data":"75620aa0eb9053783ed62e72e180e26fae7f52168e0ad2a32e04317816442665"} Jul 28 14:22:50.688870 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.688844 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" event={"ID":"6ff9d18249fb2db5835b1f8658874702","Type":"ContainerStarted","Data":"1e4e7ea8f7c50fe6b003069c6c000cd5f8496ce8091ea49f50b685a97fc9ffe1"} Jul 28 14:22:50.776219 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:50.776190 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-88.ec2.internal\" not found" Jul 28 14:22:50.779971 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.779956 2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 14:22:50.867438 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.867410 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" Jul 28 14:22:50.882732 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.882712 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 14:22:50.886114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.886101 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" Jul 28 14:22:50.896725 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.896708 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 14:22:50.991301 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:50.991224 2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 14:22:51.544256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.544186 2586 apiserver.go:52] "Watching apiserver" Jul 28 14:22:51.554726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.554454 2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 14:22:51.555533 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.555500 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal","openshift-multus/multus-45bhv","openshift-multus/multus-additional-cni-plugins-xkwwc","openshift-network-diagnostics/network-check-target-tlw8r","kube-system/konnectivity-agent-rj94j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws","openshift-cluster-node-tuning-operator/tuned-6ldkz","openshift-multus/network-metrics-daemon-9gcnr","openshift-network-operator/iptables-alerter-98sl8","openshift-ovn-kubernetes/ovnkube-node-crhnz","kube-system/global-pull-secret-syncer-69ppv","kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal","openshift-dns/node-resolver-cllq2","openshift-image-registry/node-ca-h7255"] Jul 28 14:22:51.557165 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.557116 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.559094 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.558800 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.560070 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.559891 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.560070 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.559919 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.560528 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.560507 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.561347 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.561252 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tcrx4\"" Jul 28 14:22:51.561880 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.561858 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vpwgk\"" Jul 28 14:22:51.562134 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.562117 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.562213 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.562123 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.562622 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.562320 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:51.562622 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.562390 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 14:22:51.562622 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.562390 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:22:51.563108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.562988 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.563108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.563016 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 14:22:51.563301 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.563281 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.563540 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.563367 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-chbrv\"" Jul 28 14:22:51.563540 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.563438 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 14:22:51.563540 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.563484 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.563949 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.563903 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 14:22:51.565118 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.564698 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.566415 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.566392 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 14:22:51.566604 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.566582 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.569090 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.568263 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.570137 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.570116 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.570700 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.570679 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rjqjh\"" Jul 28 14:22:51.570984 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.570933 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 14:22:51.571742 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.571720 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 14:22:51.571854 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.571781 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 14:22:51.571971 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.571949 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 14:22:51.571971 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.571966 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.572126 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.572108 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9c2vm\"" Jul 28 14:22:51.572230 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.572209 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.572326 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.572303 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.572482 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.572467 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.572666 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.572649 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.573409 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.573357 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.573957 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.573939 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l5w6j\"" Jul 28 14:22:51.574058 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.573999 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cqncs\"" Jul 28 14:22:51.574243 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.574227 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zttf7\"" Jul 28 14:22:51.574458 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.574439 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 14:22:51.574536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.574503 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.575205 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.575162 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 14:22:51.575707 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.575461 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 14:22:51.575954 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.575931 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.576050 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.576005 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:22:51.576470 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.576446 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 14:22:51.576717 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.576698 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 14:22:51.576951 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.576932 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 14:22:51.577043 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.576952 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7gvrj\"" Jul 28 14:22:51.577658 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.577627 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 14:22:51.577843 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.577805 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:51.577960 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.577940 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:22:51.578678 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578657 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 14:22:51.578844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578825 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.578844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578864 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:51.579023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578894 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-modprobe-d\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578921 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-device-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.579023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578947 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-kubernetes\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578969 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-sys\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.578995 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-os-release\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.579023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579020 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/771b4120-696d-441e-b6de-22ceb421fe5e-agent-certs\") pod \"konnectivity-agent-rj94j\" (UID: \"771b4120-696d-441e-b6de-22ceb421fe5e\") " pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579047 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-sys-fs\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579081 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-systemd\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579103 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-lib-modules\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579137 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2df07b58-919e-42f6-a7f8-f9d18088cffd-hosts-file\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579190 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cnibin\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579231 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579258 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn6zm\" (UniqueName: \"kubernetes.io/projected/58f9d7a9-55e8-4acf-8909-a946ed151c2e-kube-api-access-dn6zm\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.579328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579289 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-etc-selinux\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579353 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysctl-conf\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579396 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-host\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579436 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-tuned\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579463 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2df07b58-919e-42f6-a7f8-f9d18088cffd-tmp-dir\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579498 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-system-cni-dir\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579543 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cni-binary-copy\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579627 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-registration-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579699 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysconfig\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.579753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579741 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-socket-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579776 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqs72\" (UniqueName: \"kubernetes.io/projected/2df07b58-919e-42f6-a7f8-f9d18088cffd-kube-api-access-zqs72\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579812 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4tsl\" (UniqueName: \"kubernetes.io/projected/eef2d017-211e-4f89-9eec-9b833a95a00f-kube-api-access-j4tsl\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579866 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysctl-d\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579934 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzht4\" (UniqueName: \"kubernetes.io/projected/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-kube-api-access-vzht4\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.579979 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/771b4120-696d-441e-b6de-22ceb421fe5e-konnectivity-ca\") pod \"konnectivity-agent-rj94j\" (UID: \"771b4120-696d-441e-b6de-22ceb421fe5e\") " pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.580020 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.580054 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-run\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.580078 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-var-lib-kubelet\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.580109 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-tmp\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.580202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.580203 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.629744 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.629720 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7dt4x" Jul 28 14:22:51.637352 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.637331 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7dt4x" Jul 28 14:22:51.664377 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.664351 2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 14:22:51.681017 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.680984 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-modprobe-d\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681035 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-device-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681067 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-etc-kubernetes\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681085 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-host\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681101 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-serviceca\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681116 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-kubernetes\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681131 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-sys\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681159 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-os-release\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681187 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-modprobe-d\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681236 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-os-release\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681239 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-sys\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681239 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9kq4\" (UniqueName: \"kubernetes.io/projected/158a522e-afdc-402a-9f80-1de157fcfd83-kube-api-access-f9kq4\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681289 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-kubelet\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681315 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-hostroot\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681241 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-kubernetes\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681344 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-lib-modules\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681241 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-device-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681366 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681468 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn6zm\" (UniqueName: \"kubernetes.io/projected/58f9d7a9-55e8-4acf-8909-a946ed151c2e-kube-api-access-dn6zm\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.681522 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681484 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-lib-modules\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681542 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-etc-selinux\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681599 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-k8s-cni-cncf-io\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681629 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-etc-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681684 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-ovn\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681715 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysctl-conf\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681740 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-host\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681766 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2df07b58-919e-42f6-a7f8-f9d18088cffd-tmp-dir\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681845 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-system-cni-dir\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681873 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cni-binary-copy\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681903 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-systemd-units\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681943 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-slash\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.682016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.681999 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1da197fb-8b59-451c-b6c8-34aa60f04823-iptables-alerter-script\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682086 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682151 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2df07b58-919e-42f6-a7f8-f9d18088cffd-tmp-dir\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682171 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-host\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682174 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysctl-conf\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682213 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-system-cni-dir\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682293 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysconfig\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682327 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-etc-selinux\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682364 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-socket-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682382 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysconfig\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682423 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-kubelet\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682530 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-socket-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682573 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4tsl\" (UniqueName: \"kubernetes.io/projected/eef2d017-211e-4f89-9eec-9b833a95a00f-kube-api-access-j4tsl\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682608 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-cni-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.682728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682636 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7wg6\" (UniqueName: \"kubernetes.io/projected/7167f2b7-efb4-42ac-a941-847c75bbeb89-kube-api-access-h7wg6\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682754 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cni-binary-copy\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682775 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysctl-d\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682822 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzht4\" (UniqueName: \"kubernetes.io/projected/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-kube-api-access-vzht4\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682873 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682920 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/158a522e-afdc-402a-9f80-1de157fcfd83-cni-binary-copy\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682951 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-sysctl-d\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682953 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-conf-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.682994 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-node-log\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683019 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-cni-netd\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683045 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovnkube-script-lib\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683073 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-run\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683076 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683098 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-tmp\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683143 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-run\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683171 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683197 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683225 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovnkube-config\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683252 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-run-netns\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683288 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-var-lib-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683334 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f23822e0-3128-490e-933b-af0a29a96495-kubelet-config\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683360 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/771b4120-696d-441e-b6de-22ceb421fe5e-agent-certs\") pod \"konnectivity-agent-rj94j\" (UID: \"771b4120-696d-441e-b6de-22ceb421fe5e\") " pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683383 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683386 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-sys-fs\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683429 2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683451 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-netns\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683587 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-run-ovn-kubernetes\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683620 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh76t\" (UniqueName: \"kubernetes.io/projected/750f2fca-ecd9-498a-babe-4e519ca4e8eb-kube-api-access-vh76t\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683648 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-cni-bin\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683654 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-sys-fs\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683684 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/158a522e-afdc-402a-9f80-1de157fcfd83-multus-daemon-config\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683708 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/58f9d7a9-55e8-4acf-8909-a946ed151c2e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683709 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-multus-certs\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683768 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-systemd\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.684335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683795 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2df07b58-919e-42f6-a7f8-f9d18088cffd-hosts-file\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683819 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cnibin\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683845 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-systemd\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683858 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2df07b58-919e-42f6-a7f8-f9d18088cffd-hosts-file\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683847 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f23822e0-3128-490e-933b-af0a29a96495-dbus\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683920 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1da197fb-8b59-451c-b6c8-34aa60f04823-host-slash\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683887 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58f9d7a9-55e8-4acf-8909-a946ed151c2e-cnibin\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683951 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-tuned\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.683979 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-registration-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684006 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-system-cni-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-os-release\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684053 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-cni-multus\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684081 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovn-node-metrics-cert\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684105 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684124 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eef2d017-211e-4f89-9eec-9b833a95a00f-registration-dir\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684129 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-systemd\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684171 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqs72\" (UniqueName: \"kubernetes.io/projected/2df07b58-919e-42f6-a7f8-f9d18088cffd-kube-api-access-zqs72\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.685214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684213 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684241 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684357 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwbqz\" (UniqueName: \"kubernetes.io/projected/1da197fb-8b59-451c-b6c8-34aa60f04823-kube-api-access-cwbqz\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684386 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkdvx\" (UniqueName: \"kubernetes.io/projected/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-kube-api-access-rkdvx\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684417 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/771b4120-696d-441e-b6de-22ceb421fe5e-konnectivity-ca\") pod \"konnectivity-agent-rj94j\" (UID: \"771b4120-696d-441e-b6de-22ceb421fe5e\") " pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684476 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-cnibin\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684505 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-socket-dir-parent\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684522 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-log-socket\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684555 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-env-overrides\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684601 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-var-lib-kubelet\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684630 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684657 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-cni-bin\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684690 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.684780 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-var-lib-kubelet\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.686072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.685023 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/771b4120-696d-441e-b6de-22ceb421fe5e-konnectivity-ca\") pod \"konnectivity-agent-rj94j\" (UID: \"771b4120-696d-441e-b6de-22ceb421fe5e\") " pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.687427 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.687397 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-etc-tuned\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.687708 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.687686 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/771b4120-696d-441e-b6de-22ceb421fe5e-agent-certs\") pod \"konnectivity-agent-rj94j\" (UID: \"771b4120-696d-441e-b6de-22ceb421fe5e\") " pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.687949 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.687872 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-tmp\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.692761 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.692739 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:22:51.692858 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.692770 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:22:51.692858 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.692800 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:51.692959 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.692951 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:22:52.192883828 +0000 UTC m=+2.993761379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:51.696088 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.696064 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn6zm\" (UniqueName: \"kubernetes.io/projected/58f9d7a9-55e8-4acf-8909-a946ed151c2e-kube-api-access-dn6zm\") pod \"multus-additional-cni-plugins-xkwwc\" (UID: \"58f9d7a9-55e8-4acf-8909-a946ed151c2e\") " pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.696088 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.696075 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4tsl\" (UniqueName: \"kubernetes.io/projected/eef2d017-211e-4f89-9eec-9b833a95a00f-kube-api-access-j4tsl\") pod \"aws-ebs-csi-driver-node-jzrws\" (UID: \"eef2d017-211e-4f89-9eec-9b833a95a00f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.696443 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.696421 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqs72\" (UniqueName: \"kubernetes.io/projected/2df07b58-919e-42f6-a7f8-f9d18088cffd-kube-api-access-zqs72\") pod \"node-resolver-cllq2\" (UID: \"2df07b58-919e-42f6-a7f8-f9d18088cffd\") " pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.698233 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.698212 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzht4\" (UniqueName: \"kubernetes.io/projected/3ab37a5c-a56e-4abe-9183-6a63d1b286c6-kube-api-access-vzht4\") pod \"tuned-6ldkz\" (UID: \"3ab37a5c-a56e-4abe-9183-6a63d1b286c6\") " pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.785204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785169 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f23822e0-3128-490e-933b-af0a29a96495-kubelet-config\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785238 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-netns\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785263 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-run-ovn-kubernetes\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785296 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh76t\" (UniqueName: \"kubernetes.io/projected/750f2fca-ecd9-498a-babe-4e519ca4e8eb-kube-api-access-vh76t\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785312 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-netns\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785321 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-cni-bin\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785267 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f23822e0-3128-490e-933b-af0a29a96495-kubelet-config\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785346 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-run-ovn-kubernetes\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785370 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-cni-bin\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785380 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/158a522e-afdc-402a-9f80-1de157fcfd83-multus-daemon-config\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785405 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-multus-certs\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785430 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f23822e0-3128-490e-933b-af0a29a96495-dbus\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785484 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-multus-certs\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785527 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1da197fb-8b59-451c-b6c8-34aa60f04823-host-slash\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785578 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-system-cni-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785608 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1da197fb-8b59-451c-b6c8-34aa60f04823-host-slash\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785623 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-os-release\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785645 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-cni-multus\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785655 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-system-cni-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785672 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovn-node-metrics-cert\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785696 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f23822e0-3128-490e-933b-af0a29a96495-dbus\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785713 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785690 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-cni-multus\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785723 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-os-release\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785741 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-systemd\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785760 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785800 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.785900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785831 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwbqz\" (UniqueName: \"kubernetes.io/projected/1da197fb-8b59-451c-b6c8-34aa60f04823-kube-api-access-cwbqz\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.785841 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785857 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkdvx\" (UniqueName: \"kubernetes.io/projected/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-kube-api-access-rkdvx\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785887 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-cnibin\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785842 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785804 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-systemd\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.785921 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:22:52.285899905 +0000 UTC m=+3.086777460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785839 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.785990 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-cnibin\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786017 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-socket-dir-parent\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786012 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/158a522e-afdc-402a-9f80-1de157fcfd83-multus-daemon-config\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786056 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-log-socket\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.786834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786082 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-env-overrides\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.787448 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786992 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-log-socket\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.787448 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786116 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-socket-dir-parent\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787600 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-env-overrides\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787677 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-cni-bin\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787731 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787774 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-etc-kubernetes\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787808 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-host\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787841 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-serviceca\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787877 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9kq4\" (UniqueName: \"kubernetes.io/projected/158a522e-afdc-402a-9f80-1de157fcfd83-kube-api-access-f9kq4\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787921 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-kubelet\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787953 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-hostroot\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.787991 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-k8s-cni-cncf-io\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788025 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-etc-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788053 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-ovn\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788107 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-systemd-units\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788143 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-slash\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788178 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1da197fb-8b59-451c-b6c8-34aa60f04823-iptables-alerter-script\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788218 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-kubelet\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788252 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-cni-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.789217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788289 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7wg6\" (UniqueName: \"kubernetes.io/projected/7167f2b7-efb4-42ac-a941-847c75bbeb89-kube-api-access-h7wg6\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788327 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/158a522e-afdc-402a-9f80-1de157fcfd83-cni-binary-copy\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788358 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-conf-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788393 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-node-log\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788424 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-cni-netd\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788457 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovnkube-script-lib\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788493 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovnkube-config\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788779 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-systemd-units\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.786193 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-cni-bin\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788898 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-run-netns\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.788937 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-var-lib-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.788944 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:51.789005 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:22:52.288980399 +0000 UTC m=+3.089857949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789040 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-var-lib-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789125 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-var-lib-kubelet\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789135 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-slash\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789177 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-host-run-k8s-cni-cncf-io\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789696 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/158a522e-afdc-402a-9f80-1de157fcfd83-cni-binary-copy\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789759 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-host\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.790979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.789813 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-etc-kubernetes\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.790389 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovnkube-script-lib\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.790979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.790459 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-conf-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.790979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.790926 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-serviceca\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.791477 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791312 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-run-netns\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.791477 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791365 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-multus-cni-dir\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.791477 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791442 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovn-node-metrics-cert\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.791477 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791457 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-kubelet\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.791785 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791529 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/158a522e-afdc-402a-9f80-1de157fcfd83-hostroot\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.791785 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791599 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-etc-openvswitch\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.791896 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791782 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-run-ovn\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.791896 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791844 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-node-log\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.792004 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791884 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7167f2b7-efb4-42ac-a941-847c75bbeb89-ovnkube-config\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.792004 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.791927 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7167f2b7-efb4-42ac-a941-847c75bbeb89-host-cni-netd\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.792145 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.792104 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1da197fb-8b59-451c-b6c8-34aa60f04823-iptables-alerter-script\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.798113 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.798053 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh76t\" (UniqueName: \"kubernetes.io/projected/750f2fca-ecd9-498a-babe-4e519ca4e8eb-kube-api-access-vh76t\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:51.798397 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.798371 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwbqz\" (UniqueName: \"kubernetes.io/projected/1da197fb-8b59-451c-b6c8-34aa60f04823-kube-api-access-cwbqz\") pod \"iptables-alerter-98sl8\" (UID: \"1da197fb-8b59-451c-b6c8-34aa60f04823\") " pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.799137 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.799118 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkdvx\" (UniqueName: \"kubernetes.io/projected/97c6451f-7a7a-41c6-bbe6-fa6d6733fb05-kube-api-access-rkdvx\") pod \"node-ca-h7255\" (UID: \"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05\") " pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.800422 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.800401 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9kq4\" (UniqueName: \"kubernetes.io/projected/158a522e-afdc-402a-9f80-1de157fcfd83-kube-api-access-f9kq4\") pod \"multus-45bhv\" (UID: \"158a522e-afdc-402a-9f80-1de157fcfd83\") " pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.800555 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.800538 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7wg6\" (UniqueName: \"kubernetes.io/projected/7167f2b7-efb4-42ac-a941-847c75bbeb89-kube-api-access-h7wg6\") pod \"ovnkube-node-crhnz\" (UID: \"7167f2b7-efb4-42ac-a941-847c75bbeb89\") " pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.872485 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.872456 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" Jul 28 14:22:51.881457 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.881438 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cllq2" Jul 28 14:22:51.890181 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.890159 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" Jul 28 14:22:51.899815 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.899795 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:22:51.906523 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.906502 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" Jul 28 14:22:51.913282 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.913255 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98sl8" Jul 28 14:22:51.918877 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.918856 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h7255" Jul 28 14:22:51.925612 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.925590 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-45bhv" Jul 28 14:22:51.931367 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.931345 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:22:51.955019 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:51.954994 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 14:22:52.193388 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.193351 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:52.193610 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.193510 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:22:52.193610 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.193531 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:22:52.193610 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.193545 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:52.193777 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.193634 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:22:53.193614109 +0000 UTC m=+3.994491647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:52.293675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.293640 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:52.293827 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.293690 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:52.293871 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.293818 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:52.293907 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.293890 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:22:53.293874267 +0000 UTC m=+4.094751803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:52.293907 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.293826 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:52.293984 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:52.293923 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:22:53.293915839 +0000 UTC m=+4.094793375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:52.640726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.640693 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 14:17:52 +0000 UTC" deadline="2028-02-23 00:51:56.965763993 +0000 UTC" Jul 28 14:22:52.640726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.640725 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13786h29m4.325043741s" Jul 28 14:22:52.649992 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.649961 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f9d7a9_55e8_4acf_8909_a946ed151c2e.slice/crio-f53560516e2a61a81c6feba0addf85dd674b898df5a2987a47abe9d65b302987 WatchSource:0}: Error finding container f53560516e2a61a81c6feba0addf85dd674b898df5a2987a47abe9d65b302987: Status 404 returned error can't find the container with id f53560516e2a61a81c6feba0addf85dd674b898df5a2987a47abe9d65b302987 Jul 28 14:22:52.654111 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.654086 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef2d017_211e_4f89_9eec_9b833a95a00f.slice/crio-ceb30e9dd4881c21f5f8df7f1820eab01118fe807b800f05909a2ea0e58e8339 WatchSource:0}: Error finding container ceb30e9dd4881c21f5f8df7f1820eab01118fe807b800f05909a2ea0e58e8339: Status 404 returned error can't find the container with id ceb30e9dd4881c21f5f8df7f1820eab01118fe807b800f05909a2ea0e58e8339 Jul 28 14:22:52.656425 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.656347 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158a522e_afdc_402a_9f80_1de157fcfd83.slice/crio-fda76a97d70ecf6d19494a882f6b18456ac28a021f53c55d38ce24b906bb1ec8 WatchSource:0}: Error finding container fda76a97d70ecf6d19494a882f6b18456ac28a021f53c55d38ce24b906bb1ec8: Status 404 returned error can't find the container with id fda76a97d70ecf6d19494a882f6b18456ac28a021f53c55d38ce24b906bb1ec8 Jul 28 14:22:52.657198 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.657175 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c6451f_7a7a_41c6_bbe6_fa6d6733fb05.slice/crio-bae679b4bdab5e06b049d9ac5db9436c3c615f32d747931fce492af404296877 WatchSource:0}: Error finding container bae679b4bdab5e06b049d9ac5db9436c3c615f32d747931fce492af404296877: Status 404 returned error can't find the container with id bae679b4bdab5e06b049d9ac5db9436c3c615f32d747931fce492af404296877 Jul 28 14:22:52.658257 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.658235 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da197fb_8b59_451c_b6c8_34aa60f04823.slice/crio-4c6b69c87707baf26c3bcff012915547097975139071f33c4c273726413b8121 WatchSource:0}: Error finding container 4c6b69c87707baf26c3bcff012915547097975139071f33c4c273726413b8121: Status 404 returned error can't find the container with id 4c6b69c87707baf26c3bcff012915547097975139071f33c4c273726413b8121 Jul 28 14:22:52.659287 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.659248 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771b4120_696d_441e_b6de_22ceb421fe5e.slice/crio-fa0644a3f5200ea748f2b1030e71774dab9080a16548a6d580076dce36d48a4c WatchSource:0}: Error finding container fa0644a3f5200ea748f2b1030e71774dab9080a16548a6d580076dce36d48a4c: Status 404 returned error can't find the container with id fa0644a3f5200ea748f2b1030e71774dab9080a16548a6d580076dce36d48a4c Jul 28 14:22:52.660023 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.659988 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7167f2b7_efb4_42ac_a941_847c75bbeb89.slice/crio-f221a3414a2a6cf70fb961f22f0ecda0f89c610e1378b21cebc700987301484c WatchSource:0}: Error finding container f221a3414a2a6cf70fb961f22f0ecda0f89c610e1378b21cebc700987301484c: Status 404 returned error can't find the container with id f221a3414a2a6cf70fb961f22f0ecda0f89c610e1378b21cebc700987301484c Jul 28 14:22:52.661949 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.661913 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df07b58_919e_42f6_a7f8_f9d18088cffd.slice/crio-ab0ad3f154b580579693b0ded85edf7c2946ca7fcf38a4128fd6c85757049cd5 WatchSource:0}: Error finding container ab0ad3f154b580579693b0ded85edf7c2946ca7fcf38a4128fd6c85757049cd5: Status 404 returned error can't find the container with id ab0ad3f154b580579693b0ded85edf7c2946ca7fcf38a4128fd6c85757049cd5 Jul 28 14:22:52.664176 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:22:52.662702 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab37a5c_a56e_4abe_9183_6a63d1b286c6.slice/crio-20e3977d74437912742f1b45630771b818f8fa005a37a8c6bc198cab723fea07 WatchSource:0}: Error finding container 20e3977d74437912742f1b45630771b818f8fa005a37a8c6bc198cab723fea07: Status 404 returned error can't find the container with id 20e3977d74437912742f1b45630771b818f8fa005a37a8c6bc198cab723fea07 Jul 28 14:22:52.697112 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.696963 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rj94j" event={"ID":"771b4120-696d-441e-b6de-22ceb421fe5e","Type":"ContainerStarted","Data":"fa0644a3f5200ea748f2b1030e71774dab9080a16548a6d580076dce36d48a4c"} Jul 28 14:22:52.698116 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.698085 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-45bhv" event={"ID":"158a522e-afdc-402a-9f80-1de157fcfd83","Type":"ContainerStarted","Data":"fda76a97d70ecf6d19494a882f6b18456ac28a021f53c55d38ce24b906bb1ec8"} Jul 28 14:22:52.699675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.699653 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" event={"ID":"eef2d017-211e-4f89-9eec-9b833a95a00f","Type":"ContainerStarted","Data":"ceb30e9dd4881c21f5f8df7f1820eab01118fe807b800f05909a2ea0e58e8339"} Jul 28 14:22:52.700797 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.700773 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerStarted","Data":"f53560516e2a61a81c6feba0addf85dd674b898df5a2987a47abe9d65b302987"} Jul 28 14:22:52.701809 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.701788 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" event={"ID":"3ab37a5c-a56e-4abe-9183-6a63d1b286c6","Type":"ContainerStarted","Data":"20e3977d74437912742f1b45630771b818f8fa005a37a8c6bc198cab723fea07"} Jul 28 14:22:52.704803 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.704783 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"f221a3414a2a6cf70fb961f22f0ecda0f89c610e1378b21cebc700987301484c"} Jul 28 14:22:52.708679 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.708656 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h7255" event={"ID":"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05","Type":"ContainerStarted","Data":"bae679b4bdab5e06b049d9ac5db9436c3c615f32d747931fce492af404296877"} Jul 28 14:22:52.710070 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.710047 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cllq2" event={"ID":"2df07b58-919e-42f6-a7f8-f9d18088cffd","Type":"ContainerStarted","Data":"ab0ad3f154b580579693b0ded85edf7c2946ca7fcf38a4128fd6c85757049cd5"} Jul 28 14:22:52.712643 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:52.712599 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98sl8" event={"ID":"1da197fb-8b59-451c-b6c8-34aa60f04823","Type":"ContainerStarted","Data":"4c6b69c87707baf26c3bcff012915547097975139071f33c4c273726413b8121"} Jul 28 14:22:53.202713 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.201951 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:53.202713 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.202170 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:22:53.202713 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.202197 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:22:53.202713 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.202211 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:53.202713 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.202275 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:22:55.202256281 +0000 UTC m=+6.003133838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:53.302497 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.302419 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:53.302497 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.302500 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:53.302770 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.302666 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:53.302770 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.302742 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:22:55.302720414 +0000 UTC m=+6.103597953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:53.303264 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.303154 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:53.303264 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.303223 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:22:55.303205473 +0000 UTC m=+6.104083014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:53.641830 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.641749 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 14:17:52 +0000 UTC" deadline="2028-04-05 19:56:25.955703512 +0000 UTC" Jul 28 14:22:53.641830 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.641795 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14813h33m32.313914008s" Jul 28 14:22:53.695959 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.694847 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:53.695959 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.695031 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:22:53.695959 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.695526 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:53.695959 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.695688 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:22:53.695959 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.695794 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:53.695959 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:53.695904 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:22:53.741436 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.741395 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" event={"ID":"6ff9d18249fb2db5835b1f8658874702","Type":"ContainerStarted","Data":"2d4e872ee74b75f6f4778a75f32cbd9f46bf1465e205300fb331d964bb8dd668"} Jul 28 14:22:53.754063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.754019 2586 generic.go:356] "Generic (PLEG): container finished" podID="ae61811891b0aefce4345602b66dee13" containerID="6fa1fb8adb14fc0d254adeb233127f5c8a8326fc54ccf375dc1b4c1415915f24" exitCode=0 Jul 28 14:22:53.754244 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.754075 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" event={"ID":"ae61811891b0aefce4345602b66dee13","Type":"ContainerDied","Data":"6fa1fb8adb14fc0d254adeb233127f5c8a8326fc54ccf375dc1b4c1415915f24"} Jul 28 14:22:53.759867 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:53.759543 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-88.ec2.internal" podStartSLOduration=2.759522215 podStartE2EDuration="2.759522215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:22:53.758327104 +0000 UTC m=+4.559204665" watchObservedRunningTime="2026-07-28 14:22:53.759522215 +0000 UTC m=+4.560399779" Jul 28 14:22:54.790885 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:54.790773 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" event={"ID":"ae61811891b0aefce4345602b66dee13","Type":"ContainerStarted","Data":"67240050b66aec2b532fb59adf7d43e0a01cb35160a61cbe80e487b8e2168c03"} Jul 28 14:22:55.222883 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:55.222162 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:55.222883 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.222388 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:22:55.222883 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.222410 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:22:55.222883 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.222424 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:55.222883 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.222487 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:22:59.222466991 +0000 UTC m=+10.023344551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:55.323056 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:55.323013 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:55.323245 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:55.323089 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:55.323245 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.323236 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:55.323363 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.323301 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:22:59.323281217 +0000 UTC m=+10.124158756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:55.323436 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.323373 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:55.323436 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.323408 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:22:59.323397265 +0000 UTC m=+10.124274807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:55.685738 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:55.685704 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:55.685738 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:55.685728 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:55.685993 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.685856 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:22:55.685993 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.685945 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:22:55.686105 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:55.686018 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:55.686169 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:55.686113 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:22:57.685548 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:57.685023 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:57.685548 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:57.685178 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:22:57.685548 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:57.685198 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:57.685548 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:57.685286 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:22:57.685548 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:57.685400 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:57.685548 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:57.685502 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:22:59.263537 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:59.263404 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:59.264070 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.263623 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:22:59.264070 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.263645 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:22:59.264070 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.263662 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:59.264070 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.263732 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:23:07.263711772 +0000 UTC m=+18.064589325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:22:59.364328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:59.364253 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:59.364534 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:59.364346 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:59.364534 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.364472 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:59.364679 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.364547 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:23:07.364529787 +0000 UTC m=+18.165407323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:22:59.364679 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.364575 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:59.364679 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.364645 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:23:07.364626129 +0000 UTC m=+18.165503683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:22:59.686278 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:59.686207 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:22:59.686474 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.686318 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:22:59.686716 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:59.686698 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:22:59.686822 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.686800 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:22:59.686901 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:22:59.686872 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:22:59.687000 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:22:59.686966 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:01.685341 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:01.685304 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:01.686247 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:01.685443 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:01.686247 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:01.685305 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:01.686247 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:01.685723 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:01.686247 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:01.685802 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:01.686247 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:01.685895 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:03.685802 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:03.685772 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:03.686204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:03.685772 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:03.686204 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:03.685987 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:03.686204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:03.685772 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:03.686204 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:03.685870 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:03.686204 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:03.686076 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:05.685434 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:05.685396 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:05.685865 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:05.685397 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:05.685865 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:05.685537 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:05.685865 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:05.685396 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:05.685865 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:05.685635 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:05.685865 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:05.685703 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:07.324585 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:07.324527 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:07.325128 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.324729 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:23:07.325128 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.324756 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:23:07.325128 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.324770 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:23:07.325128 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.324839 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:23:23.324820264 +0000 UTC m=+34.125697818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:23:07.425515 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:07.425477 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:07.425515 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:07.425532 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:07.425774 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.425655 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:23:07.425774 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.425733 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:23:23.425712203 +0000 UTC m=+34.226589739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:23:07.425774 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.425665 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:23:07.425975 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.425815 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:23:23.425795708 +0000 UTC m=+34.226673243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:23:07.685142 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:07.685113 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:07.685258 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.685213 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:07.685258 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:07.685113 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:07.685258 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:07.685239 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:07.685353 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.685287 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:07.685353 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:07.685332 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:09.685895 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:09.685865 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:09.686362 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:09.685969 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:09.686362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:09.686001 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:09.686362 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:09.686103 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:09.686362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:09.686149 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:09.686362 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:09.686262 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:11.685332 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:11.685289 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:11.685744 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:11.685289 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:11.685744 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:11.685398 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:11.685744 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:11.685512 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:11.685744 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:11.685288 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:11.685744 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:11.685605 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:12.829673 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.829272 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h7255" event={"ID":"97c6451f-7a7a-41c6-bbe6-fa6d6733fb05","Type":"ContainerStarted","Data":"fbf08f616145f78b336e754d44708fbba7d3404e29fcf05bb06954917c42f9a2"} Jul 28 14:23:12.830648 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.830624 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cllq2" event={"ID":"2df07b58-919e-42f6-a7f8-f9d18088cffd","Type":"ContainerStarted","Data":"6e6107d81bf06a5f92db798e95a7e4b9ca2eba5ca2912d581d442e4a5afc9bf7"} Jul 28 14:23:12.832016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.831991 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rj94j" event={"ID":"771b4120-696d-441e-b6de-22ceb421fe5e","Type":"ContainerStarted","Data":"e6968e5cc722884f97a2808fbe7c4f8b89629c5b916d202f20a52281e333eee0"} Jul 28 14:23:12.833781 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.833754 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-45bhv" event={"ID":"158a522e-afdc-402a-9f80-1de157fcfd83","Type":"ContainerStarted","Data":"1c07c058da2efb799f3907262eb74558009431d68a1678d0479bd4cd57b580c1"} Jul 28 14:23:12.835419 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.835396 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" event={"ID":"eef2d017-211e-4f89-9eec-9b833a95a00f","Type":"ContainerStarted","Data":"260524244b492c8a96dc72356038f383c80cf9b9aa078717fe15f0f04aede497"} Jul 28 14:23:12.836824 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.836797 2586 generic.go:356] "Generic (PLEG): container finished" podID="58f9d7a9-55e8-4acf-8909-a946ed151c2e" containerID="bd634e68a9d1c909f353f921d82d85c65c1042b609cf1419ca1e84010626eb9e" exitCode=0 Jul 28 14:23:12.836901 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.836824 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerDied","Data":"bd634e68a9d1c909f353f921d82d85c65c1042b609cf1419ca1e84010626eb9e"} Jul 28 14:23:12.838295 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.838274 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" event={"ID":"3ab37a5c-a56e-4abe-9183-6a63d1b286c6","Type":"ContainerStarted","Data":"b86767324dd7f67e75e2d2d6b203ce350dc701fee9c4d39c99bb9538c48bf276"} Jul 28 14:23:12.842063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.841711 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"3555a0f1cdabcb9a0f653428fe8939e9aa56bed44529e72b2dd00e469737ab88"} Jul 28 14:23:12.842063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.841781 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"7f98aa34161a70387b009ecd724c16621270b0363fdd5d5d24c07f6856ef78f9"} Jul 28 14:23:12.842063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.841825 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"a1d1a4a5a8323d405b1f9a9f725efd16395ae65da26178a356133a4e6ac1da94"} Jul 28 14:23:12.842063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.841869 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"41a04f367b97ec2e99409ffd1dc20382c9d583b9b2cd2b6ff484184212d7a47c"} Jul 28 14:23:12.842063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.841912 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"f805f1acc4781978bba3fc999bc244239c00a607de431b004cdbfae4a4967afb"} Jul 28 14:23:12.842063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.841952 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"1e2fdae8189e84293f255fb76303b051db6e7ae333291a0c1ea5e220740a26ba"} Jul 28 14:23:12.845976 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.845939 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h7255" podStartSLOduration=7.823938232 podStartE2EDuration="22.845927532s" totalImagesPullingTime="15.0219893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:12.845632487 +0000 UTC m=+23.646510044" watchObservedRunningTime="2026-07-28 14:23:12.845927532 +0000 UTC m=+23.646805089" Jul 28 14:23:12.846080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.846003 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-88.ec2.internal" podStartSLOduration=21.845999731 podStartE2EDuration="21.845999731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:22:54.815049803 +0000 UTC m=+5.615927365" watchObservedRunningTime="2026-07-28 14:23:12.845999731 +0000 UTC m=+23.646877287" Jul 28 14:23:12.866061 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.866014 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6ldkz" podStartSLOduration=3.753566638 podStartE2EDuration="22.865998933s" totalImagesPullingTime="19.112432295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:12.865584624 +0000 UTC m=+23.666462180" watchObservedRunningTime="2026-07-28 14:23:12.865998933 +0000 UTC m=+23.666876489" Jul 28 14:23:12.931871 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.931820 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rj94j" podStartSLOduration=3.830991139 podStartE2EDuration="22.931800199s" totalImagesPullingTime="19.10080906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:12.930873662 +0000 UTC m=+23.731751217" watchObservedRunningTime="2026-07-28 14:23:12.931800199 +0000 UTC m=+23.732677732" Jul 28 14:23:12.932314 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.932287 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cllq2" podStartSLOduration=3.834113472 podStartE2EDuration="22.932279479s" totalImagesPullingTime="19.098166007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:12.911520148 +0000 UTC m=+23.712397704" watchObservedRunningTime="2026-07-28 14:23:12.932279479 +0000 UTC m=+23.733157087" Jul 28 14:23:12.945459 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.945437 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:23:12.949650 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:12.949615 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-45bhv" podStartSLOduration=3.778981296 podStartE2EDuration="22.949603043s" totalImagesPullingTime="19.170621747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:12.949523236 +0000 UTC m=+23.750400793" watchObservedRunningTime="2026-07-28 14:23:12.949603043 +0000 UTC m=+23.750480599" Jul 28 14:23:13.045241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.045162 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:23:13.045718 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.045703 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:23:13.414452 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.414415 2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 14:23:13.645931 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.645781 2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T14:23:13.414438947Z","UUID":"227a7494-25c3-4cc8-9307-b654b12308dc","Handler":null,"Name":"","Endpoint":""} Jul 28 14:23:13.647671 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.647646 2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 14:23:13.647794 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.647681 2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 14:23:13.684742 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.684712 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:13.684742 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.684732 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:13.684742 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.684743 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:13.685023 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:13.684852 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:13.685023 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:13.684963 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:13.685109 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:13.685068 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:13.845867 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.845821 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98sl8" event={"ID":"1da197fb-8b59-451c-b6c8-34aa60f04823","Type":"ContainerStarted","Data":"49107e2a01bc49da0be397006cbd71800f33b52403b4d3bc173a39e69788b47f"} Jul 28 14:23:13.847870 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.847808 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" event={"ID":"eef2d017-211e-4f89-9eec-9b833a95a00f","Type":"ContainerStarted","Data":"c82debbb6ff261fb2bde5774670fc8e2956b935998f09e517bea8b9ab5fdd5e7"} Jul 28 14:23:13.849068 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.849044 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rj94j" Jul 28 14:23:13.863490 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:13.863448 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-98sl8" podStartSLOduration=4.761757341 podStartE2EDuration="23.863434704s" totalImagesPullingTime="19.101677363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:13.863025231 +0000 UTC m=+24.663902788" watchObservedRunningTime="2026-07-28 14:23:13.863434704 +0000 UTC m=+24.664312297" Jul 28 14:23:14.853513 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:14.853425 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"ea6e65b773464e51eff5015d8c5304c12f7ff0f1d5598f5770b05ddb02b18078"} Jul 28 14:23:15.685898 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:15.685690 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:15.686074 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:15.685690 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:15.686074 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:15.685994 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:15.686074 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:15.685691 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:15.686223 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:15.686036 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:15.686223 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:15.686178 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:15.857812 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:15.857772 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" event={"ID":"eef2d017-211e-4f89-9eec-9b833a95a00f","Type":"ContainerStarted","Data":"830bd8452b93328be92d02505e25f65c617b96cc3e44e17a036e86231af623b4"} Jul 28 14:23:15.878244 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:15.878185 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzrws" podStartSLOduration=3.977703125 podStartE2EDuration="25.878162654s" totalImagesPullingTime="21.900459529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:15.877539756 +0000 UTC m=+26.678417312" watchObservedRunningTime="2026-07-28 14:23:15.878162654 +0000 UTC m=+26.679040213" Jul 28 14:23:17.685822 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.685632 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:17.686641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.685656 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:17.686641 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:17.685911 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:17.686641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.685720 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:17.686641 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:17.685970 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:17.686641 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:17.686044 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:17.867503 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.867467 2586 generic.go:356] "Generic (PLEG): container finished" podID="58f9d7a9-55e8-4acf-8909-a946ed151c2e" containerID="98f780c056c11e41b66baccd907626b97093f57b597f1ebbc5bb8dec84fb5e7f" exitCode=0 Jul 28 14:23:17.867681 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.867572 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerDied","Data":"98f780c056c11e41b66baccd907626b97093f57b597f1ebbc5bb8dec84fb5e7f"} Jul 28 14:23:17.871040 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.871018 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" event={"ID":"7167f2b7-efb4-42ac-a941-847c75bbeb89","Type":"ContainerStarted","Data":"f321d2998ba048c8903c22bae0e57b94af9e3e6a14e21db2481fc311e2b8a8c3"} Jul 28 14:23:17.871298 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.871284 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:23:17.871365 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.871351 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:23:17.886699 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.886672 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:23:17.952992 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:17.952940 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" podStartSLOduration=8.786845375 podStartE2EDuration="27.95292372s" totalImagesPullingTime="19.166078345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:17.95262227 +0000 UTC m=+28.753499826" watchObservedRunningTime="2026-07-28 14:23:17.95292372 +0000 UTC m=+28.753801276" Jul 28 14:23:18.873317 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:18.873290 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:23:18.889785 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:18.889766 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:23:19.330297 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.330263 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gcnr"] Jul 28 14:23:19.330473 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.330415 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:19.330542 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:19.330522 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:19.333214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.333185 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tlw8r"] Jul 28 14:23:19.333357 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.333291 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:19.333423 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:19.333374 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:19.333809 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.333787 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69ppv"] Jul 28 14:23:19.333910 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.333891 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:19.334007 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:19.333987 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:19.877047 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.877009 2586 generic.go:356] "Generic (PLEG): container finished" podID="58f9d7a9-55e8-4acf-8909-a946ed151c2e" containerID="e094e397ef059ad57ff2790cddf577bad59a91ca1d3d31508ad7510706c37191" exitCode=0 Jul 28 14:23:19.877396 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:19.877104 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerDied","Data":"e094e397ef059ad57ff2790cddf577bad59a91ca1d3d31508ad7510706c37191"} Jul 28 14:23:20.684786 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:20.684750 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:20.685071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:20.684797 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:20.685071 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:20.684882 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:20.685071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:20.684894 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:20.685071 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:20.684964 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:20.685071 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:20.685024 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:21.884118 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:21.883918 2586 generic.go:356] "Generic (PLEG): container finished" podID="58f9d7a9-55e8-4acf-8909-a946ed151c2e" containerID="55a8c092abfded7e95532886d7843792835eac21081d7c103097fb226613ecbd" exitCode=0 Jul 28 14:23:21.884118 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:21.884001 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerDied","Data":"55a8c092abfded7e95532886d7843792835eac21081d7c103097fb226613ecbd"} Jul 28 14:23:22.685258 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:22.685215 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:22.685258 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:22.685241 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:22.685258 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:22.685249 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:22.685539 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:22.685349 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gcnr" podUID="750f2fca-ecd9-498a-babe-4e519ca4e8eb" Jul 28 14:23:22.685539 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:22.685397 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tlw8r" podUID="def3fc9f-9aa2-46bd-a5ac-2e30448db0df" Jul 28 14:23:22.685539 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:22.685470 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ppv" podUID="f23822e0-3128-490e-933b-af0a29a96495" Jul 28 14:23:23.343041 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:23.342999 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:23.343476 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.343209 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 14:23:23.343476 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.343237 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 14:23:23.343476 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.343252 2586 projected.go:196] Error preparing data for projected volume kube-api-access-k7tbw for pod openshift-network-diagnostics/network-check-target-tlw8r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:23:23.343476 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.343326 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw podName:def3fc9f-9aa2-46bd-a5ac-2e30448db0df nodeName:}" failed. No retries permitted until 2026-07-28 14:23:55.343306794 +0000 UTC m=+66.144184331 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k7tbw" (UniqueName: "kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw") pod "network-check-target-tlw8r" (UID: "def3fc9f-9aa2-46bd-a5ac-2e30448db0df") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 14:23:23.444226 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:23.444143 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:23.444226 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:23.444201 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:23.444424 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.444403 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:23:23.444473 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.444459 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:23:55.444442648 +0000 UTC m=+66.245320196 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 14:23:23.444524 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.444405 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 14:23:23.444584 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:23.444566 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret podName:f23822e0-3128-490e-933b-af0a29a96495 nodeName:}" failed. No retries permitted until 2026-07-28 14:23:55.444536407 +0000 UTC m=+66.245413946 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret") pod "global-pull-secret-syncer-69ppv" (UID: "f23822e0-3128-490e-933b-af0a29a96495") : object "kube-system"/"original-pull-secret" not registered Jul 28 14:23:24.493158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.493126 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-88.ec2.internal" event="NodeReady" Jul 28 14:23:24.493724 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.493267 2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 14:23:24.539474 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.539405 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fkf52"] Jul 28 14:23:24.541871 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.541850 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hcw5f"] Jul 28 14:23:24.541994 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.541974 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.543727 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.543708 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.545146 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.545125 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 14:23:24.545471 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.545451 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 14:23:24.545919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.545902 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pl2dt\"" Jul 28 14:23:24.545919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.545913 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 14:23:24.546051 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.545908 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-htnvc\"" Jul 28 14:23:24.546099 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.546049 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 14:23:24.546505 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.546490 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 14:23:24.558691 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.558669 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hcw5f"] Jul 28 14:23:24.559275 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.559257 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fkf52"] Jul 28 14:23:24.654706 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.654679 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/963b1cde-5993-4b5f-b189-88240de3db8e-tmp-dir\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.654899 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.654740 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95kpk\" (UniqueName: \"kubernetes.io/projected/963b1cde-5993-4b5f-b189-88240de3db8e-kube-api-access-95kpk\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.654899 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.654770 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d8e0366-3dbf-4823-9eab-474ae1a2b4a8-cert\") pod \"ingress-canary-hcw5f\" (UID: \"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8\") " pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.654899 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.654801 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/963b1cde-5993-4b5f-b189-88240de3db8e-config-volume\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.654899 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.654841 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.654899 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.654864 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9st\" (UniqueName: \"kubernetes.io/projected/4d8e0366-3dbf-4823-9eab-474ae1a2b4a8-kube-api-access-gg9st\") pod \"ingress-canary-hcw5f\" (UID: \"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8\") " pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.685008 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.684978 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:24.685179 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.685090 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:24.685370 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.685096 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:24.687793 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.687764 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 14:23:24.687907 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.687826 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 14:23:24.687907 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.687840 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kq7zv\"" Jul 28 14:23:24.687907 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.687870 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 14:23:24.687907 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.687883 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 14:23:24.688161 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.688117 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z6ktw\"" Jul 28 14:23:24.756152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756118 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/963b1cde-5993-4b5f-b189-88240de3db8e-tmp-dir\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.756329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756184 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95kpk\" (UniqueName: \"kubernetes.io/projected/963b1cde-5993-4b5f-b189-88240de3db8e-kube-api-access-95kpk\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.756329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756216 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d8e0366-3dbf-4823-9eab-474ae1a2b4a8-cert\") pod \"ingress-canary-hcw5f\" (UID: \"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8\") " pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.756329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756257 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/963b1cde-5993-4b5f-b189-88240de3db8e-config-volume\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.756329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756305 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.756329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756327 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9st\" (UniqueName: \"kubernetes.io/projected/4d8e0366-3dbf-4823-9eab-474ae1a2b4a8-kube-api-access-gg9st\") pod \"ingress-canary-hcw5f\" (UID: \"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8\") " pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.756582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.756536 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/963b1cde-5993-4b5f-b189-88240de3db8e-tmp-dir\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.757002 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:24.756745 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:24.757002 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:24.756815 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:23:25.256798037 +0000 UTC m=+36.057675587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:23:24.757221 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.757046 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/963b1cde-5993-4b5f-b189-88240de3db8e-config-volume\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.761080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.761058 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4d8e0366-3dbf-4823-9eab-474ae1a2b4a8-cert\") pod \"ingress-canary-hcw5f\" (UID: \"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8\") " pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.773539 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.773517 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9st\" (UniqueName: \"kubernetes.io/projected/4d8e0366-3dbf-4823-9eab-474ae1a2b4a8-kube-api-access-gg9st\") pod \"ingress-canary-hcw5f\" (UID: \"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8\") " pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:24.777659 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.777619 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95kpk\" (UniqueName: \"kubernetes.io/projected/963b1cde-5993-4b5f-b189-88240de3db8e-kube-api-access-95kpk\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:24.860815 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:24.860782 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hcw5f" Jul 28 14:23:25.259817 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:25.259724 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:25.259962 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:25.259897 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:25.259962 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:25.259961 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:23:26.259945058 +0000 UTC m=+37.060822591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:23:25.860088 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:25.859873 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hcw5f"] Jul 28 14:23:25.864488 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:23:25.864463 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8e0366_3dbf_4823_9eab_474ae1a2b4a8.slice/crio-fc4e2846414f1ae9addb1bb12bf129d428ce12de3b5200dcbf935b69c7129df0 WatchSource:0}: Error finding container fc4e2846414f1ae9addb1bb12bf129d428ce12de3b5200dcbf935b69c7129df0: Status 404 returned error can't find the container with id fc4e2846414f1ae9addb1bb12bf129d428ce12de3b5200dcbf935b69c7129df0 Jul 28 14:23:25.896259 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:25.896224 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerStarted","Data":"a87cc1f0c5c3c529a37030eb14d2b36a5c6fd3bfbf5e6bece1fb54130a5f1211"} Jul 28 14:23:25.897379 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:25.897354 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hcw5f" event={"ID":"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8","Type":"ContainerStarted","Data":"fc4e2846414f1ae9addb1bb12bf129d428ce12de3b5200dcbf935b69c7129df0"} Jul 28 14:23:26.269271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:26.269231 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:26.269444 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:26.269358 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:26.269444 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:26.269410 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:23:28.269395541 +0000 UTC m=+39.070273076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:23:26.902076 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:26.902041 2586 generic.go:356] "Generic (PLEG): container finished" podID="58f9d7a9-55e8-4acf-8909-a946ed151c2e" containerID="a87cc1f0c5c3c529a37030eb14d2b36a5c6fd3bfbf5e6bece1fb54130a5f1211" exitCode=0 Jul 28 14:23:26.902439 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:26.902092 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerDied","Data":"a87cc1f0c5c3c529a37030eb14d2b36a5c6fd3bfbf5e6bece1fb54130a5f1211"} Jul 28 14:23:27.906765 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:27.906734 2586 generic.go:356] "Generic (PLEG): container finished" podID="58f9d7a9-55e8-4acf-8909-a946ed151c2e" containerID="36bc2c2c1132f6025fdade0ecbd8ab3d0d9a2a8b86a5143eea5e97506c9a1767" exitCode=0 Jul 28 14:23:27.907158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:27.906781 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerDied","Data":"36bc2c2c1132f6025fdade0ecbd8ab3d0d9a2a8b86a5143eea5e97506c9a1767"} Jul 28 14:23:28.284944 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:28.284865 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:28.285084 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:28.284991 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:28.285084 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:28.285045 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:23:32.285030574 +0000 UTC m=+43.085908107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:23:28.912291 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:28.912210 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" event={"ID":"58f9d7a9-55e8-4acf-8909-a946ed151c2e","Type":"ContainerStarted","Data":"09822b1ccb90d55df912bad33cd49ed51fe55550141db130d1f4785e0610f9c5"} Jul 28 14:23:28.936526 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:28.936482 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xkwwc" podStartSLOduration=9.62556906 podStartE2EDuration="38.936467888s" totalImagesPullingTime="29.310898828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:28.934992933 +0000 UTC m=+39.735870501" watchObservedRunningTime="2026-07-28 14:23:28.936467888 +0000 UTC m=+39.737345444" Jul 28 14:23:32.310016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:32.309837 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:32.310358 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:32.310003 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:32.310358 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:32.310125 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:23:40.310110852 +0000 UTC m=+51.110988385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:23:32.923747 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:32.923718 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hcw5f" event={"ID":"4d8e0366-3dbf-4823-9eab-474ae1a2b4a8","Type":"ContainerStarted","Data":"51d2529ab2581a6ebc86563623fc59ca98d09bebef14610b0e0b5796c3c04a95"} Jul 28 14:23:32.939191 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:32.939145 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hcw5f" podStartSLOduration=2.959897645 podStartE2EDuration="8.939130423s" totalImagesPullingTime="5.979232778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:23:32.938688106 +0000 UTC m=+43.739565662" watchObservedRunningTime="2026-07-28 14:23:32.939130423 +0000 UTC m=+43.740007979" Jul 28 14:23:40.365244 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:40.365207 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:40.365716 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:40.365357 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:40.365716 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:40.365419 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:23:56.365401534 +0000 UTC m=+67.166279068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:23:50.899114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:50.899085 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-crhnz" Jul 28 14:23:55.372033 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.371993 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:55.375159 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.375136 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 14:23:55.384868 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.384845 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 14:23:55.396744 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.396722 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7tbw\" (UniqueName: \"kubernetes.io/projected/def3fc9f-9aa2-46bd-a5ac-2e30448db0df-kube-api-access-k7tbw\") pod \"network-check-target-tlw8r\" (UID: \"def3fc9f-9aa2-46bd-a5ac-2e30448db0df\") " pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:55.472345 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.472310 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:55.472501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.472358 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:23:55.475172 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.475150 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 14:23:55.475283 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.475188 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 14:23:55.483088 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:55.483072 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 14:23:55.483172 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:55.483127 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs podName:750f2fca-ecd9-498a-babe-4e519ca4e8eb nodeName:}" failed. No retries permitted until 2026-07-28 14:24:59.483111557 +0000 UTC m=+130.283989090 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs") pod "network-metrics-daemon-9gcnr" (UID: "750f2fca-ecd9-498a-babe-4e519ca4e8eb") : secret "metrics-daemon-secret" not found Jul 28 14:23:55.485124 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.485100 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f23822e0-3128-490e-933b-af0a29a96495-original-pull-secret\") pod \"global-pull-secret-syncer-69ppv\" (UID: \"f23822e0-3128-490e-933b-af0a29a96495\") " pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:55.596119 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.596070 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ppv" Jul 28 14:23:55.606934 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.606908 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kq7zv\"" Jul 28 14:23:55.615105 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.615070 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:23:55.726881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.726852 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69ppv"] Jul 28 14:23:55.731735 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:23:55.731709 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23822e0_3128_490e_933b_af0a29a96495.slice/crio-927a0cb502d6e4ee2dbbce98ade101a4697e1ee1ca54e16731b4e98278d0a22b WatchSource:0}: Error finding container 927a0cb502d6e4ee2dbbce98ade101a4697e1ee1ca54e16731b4e98278d0a22b: Status 404 returned error can't find the container with id 927a0cb502d6e4ee2dbbce98ade101a4697e1ee1ca54e16731b4e98278d0a22b Jul 28 14:23:55.742981 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.742959 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tlw8r"] Jul 28 14:23:55.746091 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:23:55.746066 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef3fc9f_9aa2_46bd_a5ac_2e30448db0df.slice/crio-04316fe39c15d52ecc4f398b5f8554cfe28d6f9bc2c1d02055e195491b94bc60 WatchSource:0}: Error finding container 04316fe39c15d52ecc4f398b5f8554cfe28d6f9bc2c1d02055e195491b94bc60: Status 404 returned error can't find the container with id 04316fe39c15d52ecc4f398b5f8554cfe28d6f9bc2c1d02055e195491b94bc60 Jul 28 14:23:55.979082 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.979002 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tlw8r" event={"ID":"def3fc9f-9aa2-46bd-a5ac-2e30448db0df","Type":"ContainerStarted","Data":"04316fe39c15d52ecc4f398b5f8554cfe28d6f9bc2c1d02055e195491b94bc60"} Jul 28 14:23:55.980142 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:55.980120 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69ppv" event={"ID":"f23822e0-3128-490e-933b-af0a29a96495","Type":"ContainerStarted","Data":"927a0cb502d6e4ee2dbbce98ade101a4697e1ee1ca54e16731b4e98278d0a22b"} Jul 28 14:23:56.378070 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:23:56.378029 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:23:56.378536 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:56.378197 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 14:23:56.378536 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:23:56.378293 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls podName:963b1cde-5993-4b5f-b189-88240de3db8e nodeName:}" failed. No retries permitted until 2026-07-28 14:24:28.378270207 +0000 UTC m=+99.179147762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls") pod "dns-default-fkf52" (UID: "963b1cde-5993-4b5f-b189-88240de3db8e") : secret "dns-default-metrics-tls" not found Jul 28 14:24:00.996881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:00.996793 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tlw8r" event={"ID":"def3fc9f-9aa2-46bd-a5ac-2e30448db0df","Type":"ContainerStarted","Data":"f803e3749832d50c5b0fdbe732644e4c575308937e21bbee5d13b6abcdfec140"} Jul 28 14:24:00.997322 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:00.996894 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:24:00.998237 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:00.998211 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69ppv" event={"ID":"f23822e0-3128-490e-933b-af0a29a96495","Type":"ContainerStarted","Data":"3217214ba6105a0dd18934293ca490742d29944642765eb0810486ecda99ed7b"} Jul 28 14:24:01.012017 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:01.011974 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tlw8r" podStartSLOduration=66.201692733 podStartE2EDuration="1m11.011961708s" totalImagesPullingTime="4.810268975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:01.011515216 +0000 UTC m=+71.812392772" watchObservedRunningTime="2026-07-28 14:24:01.011961708 +0000 UTC m=+71.812839264" Jul 28 14:24:01.026202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:01.026161 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-69ppv" podStartSLOduration=66.200988096 podStartE2EDuration="1m11.026147463s" totalImagesPullingTime="4.825159367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:01.025343261 +0000 UTC m=+71.826220816" watchObservedRunningTime="2026-07-28 14:24:01.026147463 +0000 UTC m=+71.827025018" Jul 28 14:24:09.670252 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.670220 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg"] Jul 28 14:24:09.673059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.673045 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.676662 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.676641 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8hnfs\"" Jul 28 14:24:09.677354 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.677330 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:24:09.677354 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.677330 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 14:24:09.677503 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.677375 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 14:24:09.677703 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.677686 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 14:24:09.678178 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.678152 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns"] Jul 28 14:24:09.680871 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.680856 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.683432 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.683412 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 14:24:09.683529 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.683443 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rkt7j\"" Jul 28 14:24:09.683937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.683895 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 14:24:09.684037 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.683997 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 14:24:09.684304 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.684285 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg"] Jul 28 14:24:09.685117 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.685095 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 14:24:09.696702 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.696681 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns"] Jul 28 14:24:09.767047 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.767015 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d"] Jul 28 14:24:09.769861 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.769844 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" Jul 28 14:24:09.772991 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.772960 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.773172 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.773153 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4df310a6-b860-4f23-9e5e-e4da2c5c3618-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.773337 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.773320 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj84b\" (UniqueName: \"kubernetes.io/projected/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-kube-api-access-fj84b\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.773444 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.773430 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4df310a6-b860-4f23-9e5e-e4da2c5c3618-config\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.773616 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.773600 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh5fm\" (UniqueName: \"kubernetes.io/projected/4df310a6-b860-4f23-9e5e-e4da2c5c3618-kube-api-access-lh5fm\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.773762 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.773728 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.774273 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.774250 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jcnnr\"" Jul 28 14:24:09.774601 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.774579 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 14:24:09.774846 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.774828 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:24:09.784218 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.784194 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c95cfb5-wwnts"] Jul 28 14:24:09.787145 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.787124 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss"] Jul 28 14:24:09.787256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.787239 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.789645 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.789624 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 14:24:09.789867 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.789844 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d"] Jul 28 14:24:09.789967 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.789932 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.790425 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.790410 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 14:24:09.790529 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.790507 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 14:24:09.790625 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.790602 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xg5kp\"" Jul 28 14:24:09.793092 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.793074 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:24:09.793403 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.793386 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 14:24:09.793461 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.793420 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 14:24:09.794325 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.794198 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4flhw\"" Jul 28 14:24:09.794777 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.794762 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 14:24:09.798508 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.798490 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 14:24:09.804416 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.804395 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss"] Jul 28 14:24:09.808564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.808531 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c95cfb5-wwnts"] Jul 28 14:24:09.874405 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874374 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6xdk\" (UniqueName: \"kubernetes.io/projected/b9551376-11dc-4d39-8d3d-ef0abd3d5047-kube-api-access-w6xdk\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.874582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874417 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh5fm\" (UniqueName: \"kubernetes.io/projected/4df310a6-b860-4f23-9e5e-e4da2c5c3618-kube-api-access-lh5fm\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.874582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874436 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-certificates\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.874582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874501 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.874582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874527 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2e21f64-a578-4c74-a7bd-a271d80a92b1-ca-trust-extracted\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.874582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874544 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-trusted-ca\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.874790 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874587 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-image-registry-private-configuration\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.874790 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874632 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.874790 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:09.874741 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:09.874790 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874754 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4df310a6-b860-4f23-9e5e-e4da2c5c3618-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.874991 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874823 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj84b\" (UniqueName: \"kubernetes.io/projected/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-kube-api-access-fj84b\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.874991 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874867 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4df310a6-b860-4f23-9e5e-e4da2c5c3618-config\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.874991 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:09.874894 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls podName:139aebeb-aaf4-4581-96fd-a0ea5fc69b32 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:10.37487083 +0000 UTC m=+81.175748367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-wg8ns" (UID: "139aebeb-aaf4-4581-96fd-a0ea5fc69b32") : secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:09.874991 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.874929 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9551376-11dc-4d39-8d3d-ef0abd3d5047-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.875195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875005 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.875195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875038 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vn95\" (UniqueName: \"kubernetes.io/projected/024aa77d-658a-45b4-8a26-eca38899c269-kube-api-access-6vn95\") pod \"volume-data-source-validator-549c6d54cf-n4m4d\" (UID: \"024aa77d-658a-45b4-8a26-eca38899c269\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" Jul 28 14:24:09.875195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875086 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-bound-sa-token\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.875195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875124 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9551376-11dc-4d39-8d3d-ef0abd3d5047-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.875195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875142 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-installation-pull-secrets\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.875195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875180 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2ghd\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-kube-api-access-x2ghd\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.875488 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875307 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.875488 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.875471 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4df310a6-b860-4f23-9e5e-e4da2c5c3618-config\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.876876 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.876857 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4df310a6-b860-4f23-9e5e-e4da2c5c3618-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.883461 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.883439 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj84b\" (UniqueName: \"kubernetes.io/projected/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-kube-api-access-fj84b\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:09.883601 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.883586 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh5fm\" (UniqueName: \"kubernetes.io/projected/4df310a6-b860-4f23-9e5e-e4da2c5c3618-kube-api-access-lh5fm\") pod \"service-ca-operator-5d6c75f84f-c2jrg\" (UID: \"4df310a6-b860-4f23-9e5e-e4da2c5c3618\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.975850 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.975759 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9551376-11dc-4d39-8d3d-ef0abd3d5047-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.975850 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.975804 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.975850 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.975823 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vn95\" (UniqueName: \"kubernetes.io/projected/024aa77d-658a-45b4-8a26-eca38899c269-kube-api-access-6vn95\") pod \"volume-data-source-validator-549c6d54cf-n4m4d\" (UID: \"024aa77d-658a-45b4-8a26-eca38899c269\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" Jul 28 14:24:09.975850 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.975841 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-bound-sa-token\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.975875 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9551376-11dc-4d39-8d3d-ef0abd3d5047-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:09.975936 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:09.975959 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c95cfb5-wwnts: secret "image-registry-tls" not found Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.975998 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-installation-pull-secrets\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:09.976022 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls podName:a2e21f64-a578-4c74-a7bd-a271d80a92b1 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:10.476002528 +0000 UTC m=+81.276880087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls") pod "image-registry-c95cfb5-wwnts" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1") : secret "image-registry-tls" not found Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976055 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2ghd\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-kube-api-access-x2ghd\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976098 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6xdk\" (UniqueName: \"kubernetes.io/projected/b9551376-11dc-4d39-8d3d-ef0abd3d5047-kube-api-access-w6xdk\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.976168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976152 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-certificates\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976577 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976183 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2e21f64-a578-4c74-a7bd-a271d80a92b1-ca-trust-extracted\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976577 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976208 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-trusted-ca\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976577 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976251 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-image-registry-private-configuration\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976577 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976528 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9551376-11dc-4d39-8d3d-ef0abd3d5047-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.976729 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976664 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2e21f64-a578-4c74-a7bd-a271d80a92b1-ca-trust-extracted\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.976886 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.976870 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-certificates\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.977275 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.977251 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-trusted-ca\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.978310 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.978295 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9551376-11dc-4d39-8d3d-ef0abd3d5047-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.978748 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.978729 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-image-registry-private-configuration\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.978819 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.978772 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-installation-pull-secrets\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.982239 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.982215 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" Jul 28 14:24:09.984815 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.984792 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6xdk\" (UniqueName: \"kubernetes.io/projected/b9551376-11dc-4d39-8d3d-ef0abd3d5047-kube-api-access-w6xdk\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss\" (UID: \"b9551376-11dc-4d39-8d3d-ef0abd3d5047\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:09.985563 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.985525 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vn95\" (UniqueName: \"kubernetes.io/projected/024aa77d-658a-45b4-8a26-eca38899c269-kube-api-access-6vn95\") pod \"volume-data-source-validator-549c6d54cf-n4m4d\" (UID: \"024aa77d-658a-45b4-8a26-eca38899c269\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" Jul 28 14:24:09.985793 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.985777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2ghd\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-kube-api-access-x2ghd\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:09.985902 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:09.985885 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-bound-sa-token\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:10.081817 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.081789 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" Jul 28 14:24:10.102788 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.102760 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" Jul 28 14:24:10.104433 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.104388 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg"] Jul 28 14:24:10.106923 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:10.106897 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df310a6_b860_4f23_9e5e_e4da2c5c3618.slice/crio-b6224a40128c61d252d85b278b32eb471008a8d110fda1820cd16b0677a0fc27 WatchSource:0}: Error finding container b6224a40128c61d252d85b278b32eb471008a8d110fda1820cd16b0677a0fc27: Status 404 returned error can't find the container with id b6224a40128c61d252d85b278b32eb471008a8d110fda1820cd16b0677a0fc27 Jul 28 14:24:10.215626 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.215596 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d"] Jul 28 14:24:10.220096 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:10.220066 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024aa77d_658a_45b4_8a26_eca38899c269.slice/crio-5177f2bf00aaaaff750068451710e882a2ca8c926cf7e9ee4b747f87ea44ab14 WatchSource:0}: Error finding container 5177f2bf00aaaaff750068451710e882a2ca8c926cf7e9ee4b747f87ea44ab14: Status 404 returned error can't find the container with id 5177f2bf00aaaaff750068451710e882a2ca8c926cf7e9ee4b747f87ea44ab14 Jul 28 14:24:10.237857 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.237809 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss"] Jul 28 14:24:10.240469 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:10.240446 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9551376_11dc_4d39_8d3d_ef0abd3d5047.slice/crio-72afffac4589c00a94d8b3da60657ad58577f0805727c9398ee6c1a101026d83 WatchSource:0}: Error finding container 72afffac4589c00a94d8b3da60657ad58577f0805727c9398ee6c1a101026d83: Status 404 returned error can't find the container with id 72afffac4589c00a94d8b3da60657ad58577f0805727c9398ee6c1a101026d83 Jul 28 14:24:10.379748 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.379707 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:10.379936 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:10.379856 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:10.379936 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:10.379920 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls podName:139aebeb-aaf4-4581-96fd-a0ea5fc69b32 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:11.379905247 +0000 UTC m=+82.180782782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-wg8ns" (UID: "139aebeb-aaf4-4581-96fd-a0ea5fc69b32") : secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:10.480303 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:10.480271 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:10.480420 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:10.480388 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 14:24:10.480420 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:10.480399 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c95cfb5-wwnts: secret "image-registry-tls" not found Jul 28 14:24:10.480487 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:10.480445 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls podName:a2e21f64-a578-4c74-a7bd-a271d80a92b1 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:11.480432138 +0000 UTC m=+82.281309672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls") pod "image-registry-c95cfb5-wwnts" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1") : secret "image-registry-tls" not found Jul 28 14:24:11.023102 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:11.023059 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" event={"ID":"4df310a6-b860-4f23-9e5e-e4da2c5c3618","Type":"ContainerStarted","Data":"b6224a40128c61d252d85b278b32eb471008a8d110fda1820cd16b0677a0fc27"} Jul 28 14:24:11.024318 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:11.024291 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" event={"ID":"b9551376-11dc-4d39-8d3d-ef0abd3d5047","Type":"ContainerStarted","Data":"72afffac4589c00a94d8b3da60657ad58577f0805727c9398ee6c1a101026d83"} Jul 28 14:24:11.025422 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:11.025403 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" event={"ID":"024aa77d-658a-45b4-8a26-eca38899c269","Type":"ContainerStarted","Data":"5177f2bf00aaaaff750068451710e882a2ca8c926cf7e9ee4b747f87ea44ab14"} Jul 28 14:24:11.387818 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:11.387778 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:11.387986 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:11.387947 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:11.388069 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:11.388025 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls podName:139aebeb-aaf4-4581-96fd-a0ea5fc69b32 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:13.38800386 +0000 UTC m=+84.188881394 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-wg8ns" (UID: "139aebeb-aaf4-4581-96fd-a0ea5fc69b32") : secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:11.488928 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:11.488895 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:11.489073 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:11.489045 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 14:24:11.489073 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:11.489064 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c95cfb5-wwnts: secret "image-registry-tls" not found Jul 28 14:24:11.489138 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:11.489121 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls podName:a2e21f64-a578-4c74-a7bd-a271d80a92b1 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:13.489104498 +0000 UTC m=+84.289982061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls") pod "image-registry-c95cfb5-wwnts" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1") : secret "image-registry-tls" not found Jul 28 14:24:12.030671 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:12.030589 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" event={"ID":"024aa77d-658a-45b4-8a26-eca38899c269","Type":"ContainerStarted","Data":"34012c200cdc3e87b4d1757a70ab687b8d086b42c8df34723c431a394165c688"} Jul 28 14:24:12.047802 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:12.047745 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-n4m4d" podStartSLOduration=1.50591628 podStartE2EDuration="3.04772562s" totalImagesPullingTime="1.54180934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:12.046836298 +0000 UTC m=+82.847713854" watchObservedRunningTime="2026-07-28 14:24:12.04772562 +0000 UTC m=+82.848603177" Jul 28 14:24:13.404212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:13.404124 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:13.404692 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:13.404303 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:13.404692 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:13.404393 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls podName:139aebeb-aaf4-4581-96fd-a0ea5fc69b32 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:17.404370742 +0000 UTC m=+88.205248298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-wg8ns" (UID: "139aebeb-aaf4-4581-96fd-a0ea5fc69b32") : secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:13.505062 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:13.505022 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:13.505240 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:13.505200 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 14:24:13.505240 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:13.505223 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c95cfb5-wwnts: secret "image-registry-tls" not found Jul 28 14:24:13.505338 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:13.505285 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls podName:a2e21f64-a578-4c74-a7bd-a271d80a92b1 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:17.505268661 +0000 UTC m=+88.306146211 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls") pod "image-registry-c95cfb5-wwnts" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1") : secret "image-registry-tls" not found Jul 28 14:24:14.039583 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:14.039533 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" event={"ID":"b9551376-11dc-4d39-8d3d-ef0abd3d5047","Type":"ContainerStarted","Data":"bd1524948e8a2c090f646a82d849748a63790909646e7c971847e9298cafd81d"} Jul 28 14:24:14.041284 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:14.041258 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" event={"ID":"4df310a6-b860-4f23-9e5e-e4da2c5c3618","Type":"ContainerStarted","Data":"cfca582538c0a2f3654762e592a1ca83104fe93eadd56d45ddd2b80b13ed861b"} Jul 28 14:24:14.057942 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:14.057898 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" podStartSLOduration=2.178404863 podStartE2EDuration="5.057883376s" totalImagesPullingTime="2.879478513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:14.057434513 +0000 UTC m=+84.858312070" watchObservedRunningTime="2026-07-28 14:24:14.057883376 +0000 UTC m=+84.858760932" Jul 28 14:24:14.079117 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:14.079041 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" podStartSLOduration=1.223749941 podStartE2EDuration="5.07901931s" totalImagesPullingTime="3.855269369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:14.078112498 +0000 UTC m=+84.878990055" watchObservedRunningTime="2026-07-28 14:24:14.07901931 +0000 UTC m=+84.879896868" Jul 28 14:24:15.005392 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.005362 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lzqmb"] Jul 28 14:24:15.008212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.008197 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.011168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.011148 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wl4mt\"" Jul 28 14:24:15.011274 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.011149 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 14:24:15.012090 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.012071 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 14:24:15.012195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.012166 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 14:24:15.012296 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.012277 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 14:24:15.022770 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.022749 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzqmb"] Jul 28 14:24:15.118202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.118169 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/149e8a6d-4f81-4b73-835f-9860a0d79164-data-volume\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.118202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.118201 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.118459 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.118222 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95926\" (UniqueName: \"kubernetes.io/projected/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-api-access-95926\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.118459 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.118258 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.118459 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.118426 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/149e8a6d-4f81-4b73-835f-9860a0d79164-crio-socket\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219005 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.218970 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/149e8a6d-4f81-4b73-835f-9860a0d79164-crio-socket\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.219062 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/149e8a6d-4f81-4b73-835f-9860a0d79164-data-volume\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.219088 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.219111 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95926\" (UniqueName: \"kubernetes.io/projected/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-api-access-95926\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.219116 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/149e8a6d-4f81-4b73-835f-9860a0d79164-crio-socket\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.219141 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.219388 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.219220 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 14:24:15.219388 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.219241 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 14:24:15.219388 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.219309 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:15.719288035 +0000 UTC m=+86.520165594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : configmap "kube-rbac-proxy" not found Jul 28 14:24:15.219388 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.219333 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:15.719322628 +0000 UTC m=+86.520200163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : secret "insights-runtime-extractor-tls" not found Jul 28 14:24:15.219612 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.219440 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/149e8a6d-4f81-4b73-835f-9860a0d79164-data-volume\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.228295 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.228267 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95926\" (UniqueName: \"kubernetes.io/projected/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-api-access-95926\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.722619 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.722582 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.722619 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:15.722624 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:15.722878 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.722721 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 14:24:15.722878 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.722800 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:16.722779266 +0000 UTC m=+87.523656812 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : configmap "kube-rbac-proxy" not found Jul 28 14:24:15.722878 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.722728 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 14:24:15.722878 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:15.722871 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:16.722856105 +0000 UTC m=+87.523733642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : secret "insights-runtime-extractor-tls" not found Jul 28 14:24:16.731849 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:16.731798 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:16.731849 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:16.731852 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:16.732272 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:16.731919 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 14:24:16.732272 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:16.731970 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 14:24:16.732272 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:16.731985 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:18.73196974 +0000 UTC m=+89.532847274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : configmap "kube-rbac-proxy" not found Jul 28 14:24:16.732272 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:16.732012 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:18.732001289 +0000 UTC m=+89.532878823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : secret "insights-runtime-extractor-tls" not found Jul 28 14:24:16.772519 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:16.772490 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cllq2_2df07b58-919e-42f6-a7f8-f9d18088cffd/dns-node-resolver/0.log" Jul 28 14:24:17.436882 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:17.436840 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:17.437057 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:17.436983 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:17.437057 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:17.437048 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls podName:139aebeb-aaf4-4581-96fd-a0ea5fc69b32 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:25.437031791 +0000 UTC m=+96.237909325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-wg8ns" (UID: "139aebeb-aaf4-4581-96fd-a0ea5fc69b32") : secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:17.537808 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:17.537776 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:17.537969 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:17.537924 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 14:24:17.537969 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:17.537942 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c95cfb5-wwnts: secret "image-registry-tls" not found Jul 28 14:24:17.538048 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:17.537994 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls podName:a2e21f64-a578-4c74-a7bd-a271d80a92b1 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:25.537979719 +0000 UTC m=+96.338857277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls") pod "image-registry-c95cfb5-wwnts" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1") : secret "image-registry-tls" not found Jul 28 14:24:17.971799 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:17.971770 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h7255_97c6451f-7a7a-41c6-bbe6-fa6d6733fb05/node-ca/0.log" Jul 28 14:24:18.572723 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:18.572691 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hcw5f_4d8e0366-3dbf-4823-9eab-474ae1a2b4a8/serve-healthcheck-canary/0.log" Jul 28 14:24:18.747071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:18.747036 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:18.747071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:18.747074 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:18.747237 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:18.747157 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 14:24:18.747237 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:18.747183 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 14:24:18.747237 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:18.747219 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:22.747205446 +0000 UTC m=+93.548082980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : configmap "kube-rbac-proxy" not found Jul 28 14:24:18.747237 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:18.747233 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:22.747226564 +0000 UTC m=+93.548104098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : secret "insights-runtime-extractor-tls" not found Jul 28 14:24:22.781499 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:22.781460 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:22.782079 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:22.781510 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:22.782079 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:22.781679 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 14:24:22.782079 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:22.781680 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 14:24:22.782079 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:22.781773 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:30.781737127 +0000 UTC m=+101.582614664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : secret "insights-runtime-extractor-tls" not found Jul 28 14:24:22.782079 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:22.781828 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:30.781813258 +0000 UTC m=+101.582690796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : configmap "kube-rbac-proxy" not found Jul 28 14:24:25.502139 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:25.502107 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:25.502492 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:25.502246 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:25.502492 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:25.502316 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls podName:139aebeb-aaf4-4581-96fd-a0ea5fc69b32 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:41.502301376 +0000 UTC m=+112.303178910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-wg8ns" (UID: "139aebeb-aaf4-4581-96fd-a0ea5fc69b32") : secret "cluster-monitoring-operator-tls" not found Jul 28 14:24:25.603246 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:25.603215 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:25.605680 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:25.605649 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"image-registry-c95cfb5-wwnts\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:25.698195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:25.698167 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:25.828499 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:25.826983 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c95cfb5-wwnts"] Jul 28 14:24:25.833528 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:25.833504 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e21f64_a578_4c74_a7bd_a271d80a92b1.slice/crio-074d453b0c1f3e1be489ab6e8468fa20506cbe73cbbdcae49b237eeb3354cafa WatchSource:0}: Error finding container 074d453b0c1f3e1be489ab6e8468fa20506cbe73cbbdcae49b237eeb3354cafa: Status 404 returned error can't find the container with id 074d453b0c1f3e1be489ab6e8468fa20506cbe73cbbdcae49b237eeb3354cafa Jul 28 14:24:26.082070 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:26.082030 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" event={"ID":"a2e21f64-a578-4c74-a7bd-a271d80a92b1","Type":"ContainerStarted","Data":"64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c"} Jul 28 14:24:26.082239 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:26.082078 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" event={"ID":"a2e21f64-a578-4c74-a7bd-a271d80a92b1","Type":"ContainerStarted","Data":"074d453b0c1f3e1be489ab6e8468fa20506cbe73cbbdcae49b237eeb3354cafa"} Jul 28 14:24:26.082239 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:26.082174 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:26.109190 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:26.109131 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" podStartSLOduration=17.109110126 podStartE2EDuration="17.109110126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:26.108175859 +0000 UTC m=+96.909053414" watchObservedRunningTime="2026-07-28 14:24:26.109110126 +0000 UTC m=+96.909987664" Jul 28 14:24:28.424002 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:28.423970 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:24:28.426230 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:28.426198 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b1cde-5993-4b5f-b189-88240de3db8e-metrics-tls\") pod \"dns-default-fkf52\" (UID: \"963b1cde-5993-4b5f-b189-88240de3db8e\") " pod="openshift-dns/dns-default-fkf52" Jul 28 14:24:28.458273 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:28.458248 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pl2dt\"" Jul 28 14:24:28.465336 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:28.465315 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fkf52" Jul 28 14:24:28.591016 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:28.590984 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fkf52"] Jul 28 14:24:28.593624 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:28.593593 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963b1cde_5993_4b5f_b189_88240de3db8e.slice/crio-973175fb4c3076a6b01d8500468ce9e4dd897e494027114fa165f91dd4c081c0 WatchSource:0}: Error finding container 973175fb4c3076a6b01d8500468ce9e4dd897e494027114fa165f91dd4c081c0: Status 404 returned error can't find the container with id 973175fb4c3076a6b01d8500468ce9e4dd897e494027114fa165f91dd4c081c0 Jul 28 14:24:29.091525 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:29.091487 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fkf52" event={"ID":"963b1cde-5993-4b5f-b189-88240de3db8e","Type":"ContainerStarted","Data":"973175fb4c3076a6b01d8500468ce9e4dd897e494027114fa165f91dd4c081c0"} Jul 28 14:24:30.842806 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:30.842775 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:30.843195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:30.842813 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:30.843195 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:30.842926 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 14:24:30.843195 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:30.842974 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls podName:149e8a6d-4f81-4b73-835f-9860a0d79164 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:46.842962112 +0000 UTC m=+117.643839646 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqmb" (UID: "149e8a6d-4f81-4b73-835f-9860a0d79164") : secret "insights-runtime-extractor-tls" not found Jul 28 14:24:30.843337 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:30.843318 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/149e8a6d-4f81-4b73-835f-9860a0d79164-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:32.003264 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:32.003241 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tlw8r" Jul 28 14:24:32.102890 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:32.102857 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fkf52" event={"ID":"963b1cde-5993-4b5f-b189-88240de3db8e","Type":"ContainerStarted","Data":"4d2b45463c9624f538510e108ff5286cc0a68de801b1b62f41c2481b62b65e30"} Jul 28 14:24:32.103047 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:32.102906 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fkf52" event={"ID":"963b1cde-5993-4b5f-b189-88240de3db8e","Type":"ContainerStarted","Data":"ce203c21043e9e97975dd0d9a2d562decbf141b7e472a8ee81dc09665f4c3209"} Jul 28 14:24:32.103047 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:32.103014 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fkf52" Jul 28 14:24:32.122072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:32.122024 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fkf52" podStartSLOduration=64.92363533 podStartE2EDuration="1m8.122008057s" totalImagesPullingTime="3.198372727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:32.121171209 +0000 UTC m=+102.922048764" watchObservedRunningTime="2026-07-28 14:24:32.122008057 +0000 UTC m=+102.922885673" Jul 28 14:24:39.565637 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.565606 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c95cfb5-wwnts"] Jul 28 14:24:39.632215 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.632185 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5945b9bd66-2dr6h"] Jul 28 14:24:39.633910 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.633896 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.652893 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.652868 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5945b9bd66-2dr6h"] Jul 28 14:24:39.810225 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810198 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-bound-sa-token\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810225 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810226 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-trusted-ca\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810380 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810274 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-registry-certificates\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810380 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810296 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9nj5\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-kube-api-access-q9nj5\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810380 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810315 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-ca-trust-extracted\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810380 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810331 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-installation-pull-secrets\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810567 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810385 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-image-registry-private-configuration\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.810567 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.810402 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-registry-tls\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911189 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-registry-certificates\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911226 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9nj5\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-kube-api-access-q9nj5\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911253 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-ca-trust-extracted\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911590 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911375 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-installation-pull-secrets\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911590 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911488 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-image-registry-private-configuration\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911590 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911520 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-registry-tls\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911749 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911597 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-bound-sa-token\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911749 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911625 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-trusted-ca\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.911749 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.911692 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-ca-trust-extracted\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.912184 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.912160 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-registry-certificates\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.912497 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.912477 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-trusted-ca\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.914167 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.914149 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-registry-tls\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.914167 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.914159 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-installation-pull-secrets\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.914284 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.914225 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-image-registry-private-configuration\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.923010 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.922986 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-bound-sa-token\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.923086 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.923020 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9nj5\" (UniqueName: \"kubernetes.io/projected/9ca68c23-1bc0-447e-96b9-b236a7fe8c41-kube-api-access-q9nj5\") pod \"image-registry-5945b9bd66-2dr6h\" (UID: \"9ca68c23-1bc0-447e-96b9-b236a7fe8c41\") " pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:39.943205 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:39.943184 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:40.070812 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:40.070779 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5945b9bd66-2dr6h"] Jul 28 14:24:40.074153 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:40.074127 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca68c23_1bc0_447e_96b9_b236a7fe8c41.slice/crio-ae3899b023581d8b685a462885c2e2d66a736ca5e99b811e4204f9c0f31743e4 WatchSource:0}: Error finding container ae3899b023581d8b685a462885c2e2d66a736ca5e99b811e4204f9c0f31743e4: Status 404 returned error can't find the container with id ae3899b023581d8b685a462885c2e2d66a736ca5e99b811e4204f9c0f31743e4 Jul 28 14:24:40.128005 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:40.127977 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" event={"ID":"9ca68c23-1bc0-447e-96b9-b236a7fe8c41","Type":"ContainerStarted","Data":"cf62d82b70592e3178260f0bcdbea2a4365c1a2fc4b2e2efe5c4bb2b0dca23df"} Jul 28 14:24:40.128095 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:40.128032 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" event={"ID":"9ca68c23-1bc0-447e-96b9-b236a7fe8c41","Type":"ContainerStarted","Data":"ae3899b023581d8b685a462885c2e2d66a736ca5e99b811e4204f9c0f31743e4"} Jul 28 14:24:40.128143 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:40.128106 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:24:40.150912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:40.150860 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" podStartSLOduration=1.150844826 podStartE2EDuration="1.150844826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:40.150211532 +0000 UTC m=+110.951089089" watchObservedRunningTime="2026-07-28 14:24:40.150844826 +0000 UTC m=+110.951722408" Jul 28 14:24:41.524410 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:41.524371 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:41.526759 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:41.526739 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/139aebeb-aaf4-4581-96fd-a0ea5fc69b32-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-wg8ns\" (UID: \"139aebeb-aaf4-4581-96fd-a0ea5fc69b32\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:41.790073 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:41.789970 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" Jul 28 14:24:41.914903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:41.914776 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns"] Jul 28 14:24:41.917461 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:41.917425 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139aebeb_aaf4_4581_96fd_a0ea5fc69b32.slice/crio-077f3cafe4e6d8cce22329d57c4c7f35b8199d3815449cc673f7de781888b89b WatchSource:0}: Error finding container 077f3cafe4e6d8cce22329d57c4c7f35b8199d3815449cc673f7de781888b89b: Status 404 returned error can't find the container with id 077f3cafe4e6d8cce22329d57c4c7f35b8199d3815449cc673f7de781888b89b Jul 28 14:24:42.108868 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:42.108842 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fkf52" Jul 28 14:24:42.136162 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:42.136139 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" event={"ID":"139aebeb-aaf4-4581-96fd-a0ea5fc69b32","Type":"ContainerStarted","Data":"077f3cafe4e6d8cce22329d57c4c7f35b8199d3815449cc673f7de781888b89b"} Jul 28 14:24:44.764890 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:44.764856 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5"] Jul 28 14:24:44.787385 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:44.787360 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5"] Jul 28 14:24:44.787520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:44.787450 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:44.790640 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:44.790622 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 14:24:44.790870 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:44.790855 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-x52q9\"" Jul 28 14:24:44.950035 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:44.950000 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c54539fe-2711-4fcd-ac5e-59d4922447ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zv6f5\" (UID: \"c54539fe-2711-4fcd-ac5e-59d4922447ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:45.050602 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:45.050503 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c54539fe-2711-4fcd-ac5e-59d4922447ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zv6f5\" (UID: \"c54539fe-2711-4fcd-ac5e-59d4922447ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:45.053647 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:45.053011 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c54539fe-2711-4fcd-ac5e-59d4922447ed-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-zv6f5\" (UID: \"c54539fe-2711-4fcd-ac5e-59d4922447ed\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:45.095876 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:45.095849 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:45.146758 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:45.146725 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" event={"ID":"139aebeb-aaf4-4581-96fd-a0ea5fc69b32","Type":"ContainerStarted","Data":"526f9a4ecd6e577cfa0cd008c9e0bb4377b681a98351b91ad60865fa46edc9a9"} Jul 28 14:24:45.182929 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:45.182884 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-wg8ns" podStartSLOduration=33.846997647 podStartE2EDuration="36.18286942s" totalImagesPullingTime="2.335871773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:45.182400149 +0000 UTC m=+115.983277704" watchObservedRunningTime="2026-07-28 14:24:45.18286942 +0000 UTC m=+115.983746975" Jul 28 14:24:45.242509 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:45.242482 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5"] Jul 28 14:24:45.245804 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:45.245780 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54539fe_2711_4fcd_ac5e_59d4922447ed.slice/crio-1fd70fef9ee9ef183fcb53af885ee9115742f469cef1805e53c22905b258a7f8 WatchSource:0}: Error finding container 1fd70fef9ee9ef183fcb53af885ee9115742f469cef1805e53c22905b258a7f8: Status 404 returned error can't find the container with id 1fd70fef9ee9ef183fcb53af885ee9115742f469cef1805e53c22905b258a7f8 Jul 28 14:24:46.153816 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:46.153770 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" event={"ID":"c54539fe-2711-4fcd-ac5e-59d4922447ed","Type":"ContainerStarted","Data":"1fd70fef9ee9ef183fcb53af885ee9115742f469cef1805e53c22905b258a7f8"} Jul 28 14:24:46.866796 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:46.866759 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:46.868947 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:46.868927 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/149e8a6d-4f81-4b73-835f-9860a0d79164-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqmb\" (UID: \"149e8a6d-4f81-4b73-835f-9860a0d79164\") " pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:47.117747 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:47.117655 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzqmb" Jul 28 14:24:47.159288 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:47.159257 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" event={"ID":"c54539fe-2711-4fcd-ac5e-59d4922447ed","Type":"ContainerStarted","Data":"c4ff833445e26ded7d7910bb24ef7ef3e4961f63463489611a471838c42faaf4"} Jul 28 14:24:47.159701 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:47.159532 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:47.164921 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:47.164901 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" Jul 28 14:24:47.177048 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:47.176996 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-zv6f5" podStartSLOduration=1.995833081 podStartE2EDuration="3.176976634s" totalImagesPullingTime="1.181143553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:47.176741787 +0000 UTC m=+117.977619340" watchObservedRunningTime="2026-07-28 14:24:47.176976634 +0000 UTC m=+117.977854190" Jul 28 14:24:47.243330 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:47.243302 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzqmb"] Jul 28 14:24:47.246751 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:47.246720 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod149e8a6d_4f81_4b73_835f_9860a0d79164.slice/crio-3cb156442f098e344f0a55181c8a952950922300be8f2ca119eb1cf440ee0991 WatchSource:0}: Error finding container 3cb156442f098e344f0a55181c8a952950922300be8f2ca119eb1cf440ee0991: Status 404 returned error can't find the container with id 3cb156442f098e344f0a55181c8a952950922300be8f2ca119eb1cf440ee0991 Jul 28 14:24:48.163535 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:48.163499 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqmb" event={"ID":"149e8a6d-4f81-4b73-835f-9860a0d79164","Type":"ContainerStarted","Data":"3cb156442f098e344f0a55181c8a952950922300be8f2ca119eb1cf440ee0991"} Jul 28 14:24:49.572181 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:49.572151 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:24:52.178362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:52.178328 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqmb" event={"ID":"149e8a6d-4f81-4b73-835f-9860a0d79164","Type":"ContainerStarted","Data":"446a79c1e0e2d283cbb28657dc5464e376ffb818f115dc6fb3d92fd753b665f5"} Jul 28 14:24:55.222564 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.222499 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2"] Jul 28 14:24:55.236485 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.236453 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.241361 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.241338 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 14:24:55.241571 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.241530 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sn888\"" Jul 28 14:24:55.242388 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.242368 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 14:24:55.245836 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.245811 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 14:24:55.264394 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.264368 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2"] Jul 28 14:24:55.304084 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.304051 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n7zg9"] Jul 28 14:24:55.310726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.310697 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.313855 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.313828 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5brqd\"" Jul 28 14:24:55.313855 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.313844 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 14:24:55.314026 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.313898 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 14:24:55.314026 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.313957 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 14:24:55.333479 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333456 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91c33a8e-5d40-443d-996a-62a489c2e5e5-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.333622 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333488 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-wtmp\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.333622 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333516 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50021748-9356-4992-a3ce-fc064f6e02b0-metrics-client-ca\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.333622 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333570 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glxxc\" (UniqueName: \"kubernetes.io/projected/91c33a8e-5d40-443d-996a-62a489c2e5e5-kube-api-access-glxxc\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.333622 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333609 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-textfile\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.333835 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333645 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-root\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.333835 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333716 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.333835 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333746 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.333835 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333801 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6vw9\" (UniqueName: \"kubernetes.io/projected/50021748-9356-4992-a3ce-fc064f6e02b0-kube-api-access-w6vw9\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.334044 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333843 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.334044 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333893 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-tls\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.334044 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333913 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.334044 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.333981 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-sys\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.434611 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434565 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50021748-9356-4992-a3ce-fc064f6e02b0-metrics-client-ca\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.434611 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434619 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glxxc\" (UniqueName: \"kubernetes.io/projected/91c33a8e-5d40-443d-996a-62a489c2e5e5-kube-api-access-glxxc\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.434860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434720 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-textfile\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.434860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434750 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-root\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.434860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434787 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.434860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434810 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.434860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434826 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-root\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434864 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6vw9\" (UniqueName: \"kubernetes.io/projected/50021748-9356-4992-a3ce-fc064f6e02b0-kube-api-access-w6vw9\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434909 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:55.434922 2586 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434950 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-tls\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.434980 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435012 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-sys\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435068 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91c33a8e-5d40-443d-996a-62a489c2e5e5-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.435114 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435067 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-textfile\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435124 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-wtmp\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435262 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50021748-9356-4992-a3ce-fc064f6e02b0-metrics-client-ca\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435272 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-wtmp\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435481 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:55.435320 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-tls podName:91c33a8e-5d40-443d-996a-62a489c2e5e5 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:55.935299971 +0000 UTC m=+126.736177511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-nmzp2" (UID: "91c33a8e-5d40-443d-996a-62a489c2e5e5") : secret "openshift-state-metrics-tls" not found Jul 28 14:24:55.435481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435332 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50021748-9356-4992-a3ce-fc064f6e02b0-sys\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.435829 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435769 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91c33a8e-5d40-443d-996a-62a489c2e5e5-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.435829 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.435768 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-accelerators-collector-config\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.437745 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.437724 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-tls\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.437837 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.437775 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/50021748-9356-4992-a3ce-fc064f6e02b0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.438055 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.438038 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.450818 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.450774 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glxxc\" (UniqueName: \"kubernetes.io/projected/91c33a8e-5d40-443d-996a-62a489c2e5e5-kube-api-access-glxxc\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.452224 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.452179 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6vw9\" (UniqueName: \"kubernetes.io/projected/50021748-9356-4992-a3ce-fc064f6e02b0-kube-api-access-w6vw9\") pod \"node-exporter-n7zg9\" (UID: \"50021748-9356-4992-a3ce-fc064f6e02b0\") " pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.622233 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.622204 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n7zg9" Jul 28 14:24:55.938372 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.938296 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:55.940734 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:55.940712 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91c33a8e-5d40-443d-996a-62a489c2e5e5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-nmzp2\" (UID: \"91c33a8e-5d40-443d-996a-62a489c2e5e5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:56.148075 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.148042 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" Jul 28 14:24:56.365847 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.365819 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:24:56.367584 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:56.367540 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50021748_9356_4992_a3ce_fc064f6e02b0.slice/crio-a1487ad2d38316c4daaadecef8fe2b7baf1436cd7abe1c764a37b87c8f4cd510 WatchSource:0}: Error finding container a1487ad2d38316c4daaadecef8fe2b7baf1436cd7abe1c764a37b87c8f4cd510: Status 404 returned error can't find the container with id a1487ad2d38316c4daaadecef8fe2b7baf1436cd7abe1c764a37b87c8f4cd510 Jul 28 14:24:56.370638 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.369966 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.379881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.379725 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 14:24:56.379881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.379763 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 14:24:56.379881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.379770 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 14:24:56.380099 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.380027 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 14:24:56.380099 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.380060 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 14:24:56.380205 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.380140 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wgpzt\"" Jul 28 14:24:56.380205 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.380173 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 14:24:56.380460 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.380442 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 14:24:56.380579 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.380515 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 14:24:56.381254 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.381238 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 14:24:56.400953 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.400928 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:24:56.440981 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.440951 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.440998 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-292lg\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-kube-api-access-292lg\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441034 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441057 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441140 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441165 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441204 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441187 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441269 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441244 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441317 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441299 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441353 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441335 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441387 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441375 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441428 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441403 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-config-out\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.441484 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.441442 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-web-config\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.501582 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.501538 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2"] Jul 28 14:24:56.504737 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:56.504712 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c33a8e_5d40_443d_996a_62a489c2e5e5.slice/crio-96a77b4253c06320208cec131cd5025b97ff54d39a4e92677ec9a5443b8cc7a6 WatchSource:0}: Error finding container 96a77b4253c06320208cec131cd5025b97ff54d39a4e92677ec9a5443b8cc7a6: Status 404 returned error can't find the container with id 96a77b4253c06320208cec131cd5025b97ff54d39a4e92677ec9a5443b8cc7a6 Jul 28 14:24:56.542034 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542012 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.542141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542047 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.542141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542068 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.542251 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542170 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.542251 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542218 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542611 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542687 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542735 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-config-out\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542786 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-web-config\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542830 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542952 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.543937 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:24:56.543453 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle podName:f4d59513-da52-4f5a-8f96-00528f860f26 nodeName:}" failed. No retries permitted until 2026-07-28 14:24:57.043427206 +0000 UTC m=+127.844304768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26") : configmap references non-existent config key: ca-bundle.crt Jul 28 14:24:56.545726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.542865 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-292lg\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-kube-api-access-292lg\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.545891 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.545783 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.545891 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.545819 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.546753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.546730 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.547235 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.547210 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-web-config\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.549852 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.549799 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.554266 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.553822 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-config-out\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.554266 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.553946 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.554266 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.554002 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.555241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.555140 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.555241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.555161 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.555365 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.555355 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.556203 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.556184 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-config-volume\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:56.570831 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:56.570812 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-292lg\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-kube-api-access-292lg\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:57.050231 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.050147 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:57.050874 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.050852 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:57.198829 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.198792 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqmb" event={"ID":"149e8a6d-4f81-4b73-835f-9860a0d79164","Type":"ContainerStarted","Data":"9e2d50b655a3bb6b741bf609fca76897f63d3ee938fe229e2b604611efd8d1f3"} Jul 28 14:24:57.200793 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.200763 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" event={"ID":"91c33a8e-5d40-443d-996a-62a489c2e5e5","Type":"ContainerStarted","Data":"136c1dcb747947f4da7ed109df302ed7a2defea4d02cec3910cc4662021b94c5"} Jul 28 14:24:57.200930 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.200819 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" event={"ID":"91c33a8e-5d40-443d-996a-62a489c2e5e5","Type":"ContainerStarted","Data":"780917b6d3e6262b5b3f77374c7bb9145499a23c81c8c04992c64eba993e8df3"} Jul 28 14:24:57.200930 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.200846 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" event={"ID":"91c33a8e-5d40-443d-996a-62a489c2e5e5","Type":"ContainerStarted","Data":"96a77b4253c06320208cec131cd5025b97ff54d39a4e92677ec9a5443b8cc7a6"} Jul 28 14:24:57.202276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.202251 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n7zg9" event={"ID":"50021748-9356-4992-a3ce-fc064f6e02b0","Type":"ContainerStarted","Data":"a1487ad2d38316c4daaadecef8fe2b7baf1436cd7abe1c764a37b87c8f4cd510"} Jul 28 14:24:57.281637 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.281357 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:24:57.328675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.328237 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f946987f7-jcj4h"] Jul 28 14:24:57.334151 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.333679 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.341302 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.341279 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 14:24:57.341508 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.341489 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 14:24:57.341789 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.341719 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vwt8q\"" Jul 28 14:24:57.342104 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.341958 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 14:24:57.342104 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.342025 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 14:24:57.342350 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.342153 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-69b62fghf31rl\"" Jul 28 14:24:57.342350 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.342298 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 14:24:57.348246 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.348220 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f946987f7-jcj4h"] Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352741 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352821 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plswb\" (UniqueName: \"kubernetes.io/projected/6bedff0c-1136-4a44-a5ef-c88aed387a56-kube-api-access-plswb\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352849 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352866 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-grpc-tls\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352907 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352942 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352973 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-tls\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.355916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.352992 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bedff0c-1136-4a44-a5ef-c88aed387a56-metrics-client-ca\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454335 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454301 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454373 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454426 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-tls\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454454 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bedff0c-1136-4a44-a5ef-c88aed387a56-metrics-client-ca\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454514 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454598 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plswb\" (UniqueName: \"kubernetes.io/projected/6bedff0c-1136-4a44-a5ef-c88aed387a56-kube-api-access-plswb\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454630 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.454782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.454656 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-grpc-tls\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.455667 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.455635 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6bedff0c-1136-4a44-a5ef-c88aed387a56-metrics-client-ca\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.457692 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.457449 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.457692 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.457651 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.458969 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.458920 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-grpc-tls\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.459075 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.458996 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-tls\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.459604 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.459582 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.460021 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.459981 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6bedff0c-1136-4a44-a5ef-c88aed387a56-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.464576 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.464536 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plswb\" (UniqueName: \"kubernetes.io/projected/6bedff0c-1136-4a44-a5ef-c88aed387a56-kube-api-access-plswb\") pod \"thanos-querier-7f946987f7-jcj4h\" (UID: \"6bedff0c-1136-4a44-a5ef-c88aed387a56\") " pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.474095 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.474008 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:24:57.646767 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:57.646671 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:24:57.923128 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:57.923056 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d59513_da52_4f5a_8f96_00528f860f26.slice/crio-9359302d849b549598bf9c38bc521099616c0105b7d5e344690f799e01ae9ee1 WatchSource:0}: Error finding container 9359302d849b549598bf9c38bc521099616c0105b7d5e344690f799e01ae9ee1: Status 404 returned error can't find the container with id 9359302d849b549598bf9c38bc521099616c0105b7d5e344690f799e01ae9ee1 Jul 28 14:24:58.141926 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.141891 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f946987f7-jcj4h"] Jul 28 14:24:58.144141 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:24:58.144111 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bedff0c_1136_4a44_a5ef_c88aed387a56.slice/crio-78ca4d156e3fc0d2c02d4118053d565c8d06b816912ba4ebd251d5a7538b9d22 WatchSource:0}: Error finding container 78ca4d156e3fc0d2c02d4118053d565c8d06b816912ba4ebd251d5a7538b9d22: Status 404 returned error can't find the container with id 78ca4d156e3fc0d2c02d4118053d565c8d06b816912ba4ebd251d5a7538b9d22 Jul 28 14:24:58.207768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.207699 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"78ca4d156e3fc0d2c02d4118053d565c8d06b816912ba4ebd251d5a7538b9d22"} Jul 28 14:24:58.210171 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.210143 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" event={"ID":"91c33a8e-5d40-443d-996a-62a489c2e5e5","Type":"ContainerStarted","Data":"f37ff0e8e692a4a9334790c4ec32304b6d5bfd5deb2a9cb5db6575c6caf5a853"} Jul 28 14:24:58.211496 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.211472 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"9359302d849b549598bf9c38bc521099616c0105b7d5e344690f799e01ae9ee1"} Jul 28 14:24:58.213291 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.213260 2586 generic.go:356] "Generic (PLEG): container finished" podID="50021748-9356-4992-a3ce-fc064f6e02b0" containerID="94653b04ac1abc5a9e88d2960b017e2b8f385f74c84f81264d43a14db619a9d6" exitCode=0 Jul 28 14:24:58.213490 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.213326 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n7zg9" event={"ID":"50021748-9356-4992-a3ce-fc064f6e02b0","Type":"ContainerDied","Data":"94653b04ac1abc5a9e88d2960b017e2b8f385f74c84f81264d43a14db619a9d6"} Jul 28 14:24:58.232695 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:58.232657 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-nmzp2" podStartSLOduration=1.882255415 podStartE2EDuration="3.232642362s" totalImagesPullingTime="1.350386947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:58.231546151 +0000 UTC m=+129.032423710" watchObservedRunningTime="2026-07-28 14:24:58.232642362 +0000 UTC m=+129.033519917" Jul 28 14:24:59.219832 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.219775 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n7zg9" event={"ID":"50021748-9356-4992-a3ce-fc064f6e02b0","Type":"ContainerStarted","Data":"8518661fc5aee4cb11209efd039ad0001782cd3ed8759ec9ab00570a0fb4e3aa"} Jul 28 14:24:59.220291 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.219841 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n7zg9" event={"ID":"50021748-9356-4992-a3ce-fc064f6e02b0","Type":"ContainerStarted","Data":"be16fa68b01f6d89588dc9e48b64f704b052581ac8d7a8455b7ea3fdfcd92642"} Jul 28 14:24:59.246615 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.246538 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n7zg9" podStartSLOduration=2.553168535 podStartE2EDuration="4.246522014s" totalImagesPullingTime="1.611617272s" totalInitContainerRuntime="81.736207ms" isStatefulPod=true podCreationTimestamp="2026-07-28 14:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:24:59.244843122 +0000 UTC m=+130.045720688" watchObservedRunningTime="2026-07-28 14:24:59.246522014 +0000 UTC m=+130.047399569" Jul 28 14:24:59.574734 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.574703 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:24:59.577158 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.577132 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750f2fca-ecd9-498a-babe-4e519ca4e8eb-metrics-certs\") pod \"network-metrics-daemon-9gcnr\" (UID: \"750f2fca-ecd9-498a-babe-4e519ca4e8eb\") " pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:24:59.654518 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.654479 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7469fb649c-dhfdp"] Jul 28 14:24:59.669925 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.669898 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7469fb649c-dhfdp"] Jul 28 14:24:59.670080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.670028 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.673290 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.673265 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9c8o452n2kqg1\"" Jul 28 14:24:59.674593 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.674127 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 14:24:59.674593 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.674130 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 14:24:59.674593 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.674462 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gcshn\"" Jul 28 14:24:59.674812 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.674623 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 14:24:59.674812 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.674697 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776173 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-secret-metrics-server-client-certs\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776237 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kg2s\" (UniqueName: \"kubernetes.io/projected/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-kube-api-access-2kg2s\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776276 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-client-ca-bundle\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776331 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-secret-metrics-server-tls\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776354 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776377 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-metrics-server-audit-profiles\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.776454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.776410 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-audit-log\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.812275 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.812250 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z6ktw\"" Jul 28 14:24:59.820260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.820240 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gcnr" Jul 28 14:24:59.877526 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877488 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-secret-metrics-server-client-certs\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.877720 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877583 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kg2s\" (UniqueName: \"kubernetes.io/projected/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-kube-api-access-2kg2s\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.877720 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877633 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-client-ca-bundle\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.877844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877718 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-secret-metrics-server-tls\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.877844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877752 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.877844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877783 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-metrics-server-audit-profiles\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.877844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.877821 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-audit-log\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.878299 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.878272 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-audit-log\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.878917 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.878864 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.879165 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.879141 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-metrics-server-audit-profiles\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.880768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.880742 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-secret-metrics-server-tls\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.880881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.880813 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-secret-metrics-server-client-certs\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.881398 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.881376 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-client-ca-bundle\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.892326 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.892280 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kg2s\" (UniqueName: \"kubernetes.io/projected/5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4-kube-api-access-2kg2s\") pod \"metrics-server-7469fb649c-dhfdp\" (UID: \"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4\") " pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.976149 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.976120 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j"] Jul 28 14:24:59.981601 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.981574 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:24:59.982706 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.982687 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:24:59.984281 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.984262 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-64wvk\"" Jul 28 14:24:59.984414 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.984393 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 14:24:59.990680 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:24:59.989418 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j"] Jul 28 14:25:00.079841 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.079759 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4xgq\" (UniqueName: \"kubernetes.io/projected/684319de-63bb-4f34-bd35-734c0fcd8b2a-kube-api-access-n4xgq\") pod \"monitoring-plugin-68b7fc9574-2md2j\" (UID: \"684319de-63bb-4f34-bd35-734c0fcd8b2a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.079841 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.079800 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/684319de-63bb-4f34-bd35-734c0fcd8b2a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2md2j\" (UID: \"684319de-63bb-4f34-bd35-734c0fcd8b2a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.180518 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.180483 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/684319de-63bb-4f34-bd35-734c0fcd8b2a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2md2j\" (UID: \"684319de-63bb-4f34-bd35-734c0fcd8b2a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.180706 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.180616 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4xgq\" (UniqueName: \"kubernetes.io/projected/684319de-63bb-4f34-bd35-734c0fcd8b2a-kube-api-access-n4xgq\") pod \"monitoring-plugin-68b7fc9574-2md2j\" (UID: \"684319de-63bb-4f34-bd35-734c0fcd8b2a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.183329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.183304 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/684319de-63bb-4f34-bd35-734c0fcd8b2a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2md2j\" (UID: \"684319de-63bb-4f34-bd35-734c0fcd8b2a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.190433 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.190410 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4xgq\" (UniqueName: \"kubernetes.io/projected/684319de-63bb-4f34-bd35-734c0fcd8b2a-kube-api-access-n4xgq\") pod \"monitoring-plugin-68b7fc9574-2md2j\" (UID: \"684319de-63bb-4f34-bd35-734c0fcd8b2a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.294884 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.294855 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:00.841777 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:00.841513 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gcnr"] Jul 28 14:25:00.875515 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:25:00.875480 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750f2fca_ecd9_498a_babe_4e519ca4e8eb.slice/crio-94bb640f39ea269a11c98c51310d2ae3b43460a284920bb391bbd9d3c0fdacec WatchSource:0}: Error finding container 94bb640f39ea269a11c98c51310d2ae3b43460a284920bb391bbd9d3c0fdacec: Status 404 returned error can't find the container with id 94bb640f39ea269a11c98c51310d2ae3b43460a284920bb391bbd9d3c0fdacec Jul 28 14:25:01.064136 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.064106 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j"] Jul 28 14:25:01.066972 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.066935 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7469fb649c-dhfdp"] Jul 28 14:25:01.067198 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:25:01.067171 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684319de_63bb_4f34_bd35_734c0fcd8b2a.slice/crio-6d3827b7c9a1fba10ec999056f4d4f74a6abf1ca9165e4ca450edc3f55c8bfbb WatchSource:0}: Error finding container 6d3827b7c9a1fba10ec999056f4d4f74a6abf1ca9165e4ca450edc3f55c8bfbb: Status 404 returned error can't find the container with id 6d3827b7c9a1fba10ec999056f4d4f74a6abf1ca9165e4ca450edc3f55c8bfbb Jul 28 14:25:01.069486 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:25:01.069467 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aaefdc4_e72b_419e_9d4a_7c89c50aa3a4.slice/crio-f90e94b956ad5b6d3b0c6a96e3d58e2128213c588e2a07e23f01372ac260edd8 WatchSource:0}: Error finding container f90e94b956ad5b6d3b0c6a96e3d58e2128213c588e2a07e23f01372ac260edd8: Status 404 returned error can't find the container with id f90e94b956ad5b6d3b0c6a96e3d58e2128213c588e2a07e23f01372ac260edd8 Jul 28 14:25:01.137948 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.137886 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5945b9bd66-2dr6h" Jul 28 14:25:01.229715 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.229677 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gcnr" event={"ID":"750f2fca-ecd9-498a-babe-4e519ca4e8eb","Type":"ContainerStarted","Data":"94bb640f39ea269a11c98c51310d2ae3b43460a284920bb391bbd9d3c0fdacec"} Jul 28 14:25:01.232008 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.231977 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqmb" event={"ID":"149e8a6d-4f81-4b73-835f-9860a0d79164","Type":"ContainerStarted","Data":"19b0f92a4efe2c9f8093d970439b69bf9351653375a7145dfc5527ce6773ec6c"} Jul 28 14:25:01.234164 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.234130 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344" exitCode=0 Jul 28 14:25:01.234282 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.234214 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344"} Jul 28 14:25:01.235588 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.235517 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" event={"ID":"684319de-63bb-4f34-bd35-734c0fcd8b2a","Type":"ContainerStarted","Data":"6d3827b7c9a1fba10ec999056f4d4f74a6abf1ca9165e4ca450edc3f55c8bfbb"} Jul 28 14:25:01.236856 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.236825 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" event={"ID":"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4","Type":"ContainerStarted","Data":"f90e94b956ad5b6d3b0c6a96e3d58e2128213c588e2a07e23f01372ac260edd8"} Jul 28 14:25:01.252394 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:01.252351 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lzqmb" podStartSLOduration=33.941829334 podStartE2EDuration="47.252336942s" totalImagesPullingTime="13.310507608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 14:24:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:01.251484515 +0000 UTC m=+132.052362083" watchObservedRunningTime="2026-07-28 14:25:01.252336942 +0000 UTC m=+132.053214497" Jul 28 14:25:02.663472 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.663439 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-chxst"] Jul 28 14:25:02.669831 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.669811 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:02.672400 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.672376 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 14:25:02.672524 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.672418 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5np7x\"" Jul 28 14:25:02.672524 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.672432 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 14:25:02.678526 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.678504 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-chxst"] Jul 28 14:25:02.707289 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.707265 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdnzk\" (UniqueName: \"kubernetes.io/projected/4770e716-7472-4083-a75a-35a78ada6fb4-kube-api-access-pdnzk\") pod \"downloads-7c97dc4555-chxst\" (UID: \"4770e716-7472-4083-a75a-35a78ada6fb4\") " pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:02.808777 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.808742 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdnzk\" (UniqueName: \"kubernetes.io/projected/4770e716-7472-4083-a75a-35a78ada6fb4-kube-api-access-pdnzk\") pod \"downloads-7c97dc4555-chxst\" (UID: \"4770e716-7472-4083-a75a-35a78ada6fb4\") " pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:02.818439 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.818404 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdnzk\" (UniqueName: \"kubernetes.io/projected/4770e716-7472-4083-a75a-35a78ada6fb4-kube-api-access-pdnzk\") pod \"downloads-7c97dc4555-chxst\" (UID: \"4770e716-7472-4083-a75a-35a78ada6fb4\") " pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:02.981029 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:02.980903 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:03.518004 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:03.517971 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-chxst"] Jul 28 14:25:03.760650 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:25:03.760617 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4770e716_7472_4083_a75a_35a78ada6fb4.slice/crio-386d5a517f58103747d6998b06b1d75bfad969010240b44ef1b65e60342810db WatchSource:0}: Error finding container 386d5a517f58103747d6998b06b1d75bfad969010240b44ef1b65e60342810db: Status 404 returned error can't find the container with id 386d5a517f58103747d6998b06b1d75bfad969010240b44ef1b65e60342810db Jul 28 14:25:04.251907 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.251826 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-chxst" event={"ID":"4770e716-7472-4083-a75a-35a78ada6fb4","Type":"ContainerStarted","Data":"386d5a517f58103747d6998b06b1d75bfad969010240b44ef1b65e60342810db"} Jul 28 14:25:04.253534 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.253503 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gcnr" event={"ID":"750f2fca-ecd9-498a-babe-4e519ca4e8eb","Type":"ContainerStarted","Data":"a103852118358f94bf85b28914e06cf15704a3944a70c73db9164c09cf23b058"} Jul 28 14:25:04.253663 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.253567 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gcnr" event={"ID":"750f2fca-ecd9-498a-babe-4e519ca4e8eb","Type":"ContainerStarted","Data":"14f43e67302bd120d140aeb67ceac137a24bc9d3f53e82927270aeb95c9c7edf"} Jul 28 14:25:04.256601 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.256581 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3"} Jul 28 14:25:04.256726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.256634 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497"} Jul 28 14:25:04.256726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.256661 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24"} Jul 28 14:25:04.256726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.256699 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18"} Jul 28 14:25:04.256726 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.256726 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da"} Jul 28 14:25:04.258112 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.258090 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" event={"ID":"684319de-63bb-4f34-bd35-734c0fcd8b2a","Type":"ContainerStarted","Data":"486e1c95d07f3619dd3cb4b995d5ad57b9960077e1931b464a99a482db728ad5"} Jul 28 14:25:04.258240 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.258225 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:04.260393 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.260362 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"66e9a201ef88170a7da2cda3142ddf4b14d69883556cbdd5f16ecdb8643d6d31"} Jul 28 14:25:04.260493 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.260437 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"872fb94fc5a04d35c116b7f1bac188043639999edf543dc5ed76492ca460fae2"} Jul 28 14:25:04.260493 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.260480 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"69d5a4168b0057675b73abf3aed48da4c1527e638b4743b3598a6d1c5f593646"} Jul 28 14:25:04.261945 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.261926 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" event={"ID":"5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4","Type":"ContainerStarted","Data":"ca7b7cf28684251fcef49d6d20bb904525d65e599b62eee928803fd2d7703bf2"} Jul 28 14:25:04.264297 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.264280 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" Jul 28 14:25:04.271754 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.271716 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9gcnr" podStartSLOduration=131.355370916 podStartE2EDuration="2m14.271700625s" totalImagesPullingTime="2.916329709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:04.270924829 +0000 UTC m=+135.071802386" watchObservedRunningTime="2026-07-28 14:25:04.271700625 +0000 UTC m=+135.072578184" Jul 28 14:25:04.289152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.289109 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" podStartSLOduration=2.625210935 podStartE2EDuration="5.289095543s" totalImagesPullingTime="2.663884608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:04.288647908 +0000 UTC m=+135.089525464" watchObservedRunningTime="2026-07-28 14:25:04.289095543 +0000 UTC m=+135.089973098" Jul 28 14:25:04.305491 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.305456 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2md2j" podStartSLOduration=2.589428255 podStartE2EDuration="5.30544502s" totalImagesPullingTime="2.716016765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:04.304983082 +0000 UTC m=+135.105860647" watchObservedRunningTime="2026-07-28 14:25:04.30544502 +0000 UTC m=+135.106322576" Jul 28 14:25:04.583788 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.583742 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" podUID="a2e21f64-a578-4c74-a7bd-a271d80a92b1" containerName="registry" containerID="cri-o://64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c" gracePeriod=30 Jul 28 14:25:04.903020 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:04.902978 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:25:05.026359 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026331 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2e21f64-a578-4c74-a7bd-a271d80a92b1-ca-trust-extracted\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026486 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026379 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-bound-sa-token\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026486 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026426 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-image-registry-private-configuration\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026486 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026455 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-certificates\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026676 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026494 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026676 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026545 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-installation-pull-secrets\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026676 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026610 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-trusted-ca\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.026676 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.026669 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2ghd\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-kube-api-access-x2ghd\") pod \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\" (UID: \"a2e21f64-a578-4c74-a7bd-a271d80a92b1\") " Jul 28 14:25:05.027693 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.027290 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:25:05.027693 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.027434 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:25:05.029421 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.029386 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:25:05.029654 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.029618 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:25:05.029957 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.029933 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-kube-api-access-x2ghd" (OuterVolumeSpecName: "kube-api-access-x2ghd") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "kube-api-access-x2ghd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:25:05.029957 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.029946 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:25:05.030080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.029998 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:25:05.035784 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.035763 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e21f64-a578-4c74-a7bd-a271d80a92b1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a2e21f64-a578-4c74-a7bd-a271d80a92b1" (UID: "a2e21f64-a578-4c74-a7bd-a271d80a92b1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127342 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-trusted-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127365 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2ghd\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-kube-api-access-x2ghd\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127375 2586 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2e21f64-a578-4c74-a7bd-a271d80a92b1-ca-trust-extracted\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127383 2586 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-bound-sa-token\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127393 2586 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-image-registry-private-configuration\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127402 2586 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-certificates\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127412 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127409 2586 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2e21f64-a578-4c74-a7bd-a271d80a92b1-registry-tls\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.127786 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.127417 2586 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2e21f64-a578-4c74-a7bd-a271d80a92b1-installation-pull-secrets\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:25:05.268002 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.267967 2586 generic.go:356] "Generic (PLEG): container finished" podID="a2e21f64-a578-4c74-a7bd-a271d80a92b1" containerID="64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c" exitCode=0 Jul 28 14:25:05.268159 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.268021 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" Jul 28 14:25:05.268159 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.268065 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" event={"ID":"a2e21f64-a578-4c74-a7bd-a271d80a92b1","Type":"ContainerDied","Data":"64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c"} Jul 28 14:25:05.268159 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.268133 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c95cfb5-wwnts" event={"ID":"a2e21f64-a578-4c74-a7bd-a271d80a92b1","Type":"ContainerDied","Data":"074d453b0c1f3e1be489ab6e8468fa20506cbe73cbbdcae49b237eeb3354cafa"} Jul 28 14:25:05.268330 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.268160 2586 scope.go:117] "RemoveContainer" containerID="64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c" Jul 28 14:25:05.273689 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.273659 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerStarted","Data":"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8"} Jul 28 14:25:05.284789 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.284645 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"c8ac6dbf9e91cb5e71d9196b19fa62af6bf349918a5c8de2e820e8fe65db3926"} Jul 28 14:25:05.284789 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.284751 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"2db84c32fc9549f0a95f746ec424cef5d3651751a230058290972d5df8171f79"} Jul 28 14:25:05.284903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.284833 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:25:05.284903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.284878 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" event={"ID":"6bedff0c-1136-4a44-a5ef-c88aed387a56","Type":"ContainerStarted","Data":"0485d6d7c902d7d0773f08629da44ae550ab166ceada1aff71edf412883695d6"} Jul 28 14:25:05.284903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.284885 2586 scope.go:117] "RemoveContainer" containerID="64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c" Jul 28 14:25:05.285153 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:25:05.285124 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c\": container with ID starting with 64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c not found: ID does not exist" containerID="64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c" Jul 28 14:25:05.285231 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.285159 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c"} err="failed to get container status \"64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c\": rpc error: code = NotFound desc = could not find container \"64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c\": container with ID starting with 64ca6b58d65370c0175791e308e584f62eb10f68f1728dc1fd6747d759dc0f5c not found: ID does not exist" Jul 28 14:25:05.315108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.315058 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.368168397 podStartE2EDuration="9.315038837s" totalImagesPullingTime="5.922196791s" totalInitContainerRuntime="24.673649ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:05.312706471 +0000 UTC m=+136.113584041" watchObservedRunningTime="2026-07-28 14:25:05.315038837 +0000 UTC m=+136.115916394" Jul 28 14:25:05.342006 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.341960 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c95cfb5-wwnts"] Jul 28 14:25:05.350203 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.350180 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c95cfb5-wwnts"] Jul 28 14:25:05.380605 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.380295 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" podStartSLOduration=1.991079591 podStartE2EDuration="8.380274272s" totalImagesPullingTime="6.389194681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:24:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:05.378523852 +0000 UTC m=+136.179401409" watchObservedRunningTime="2026-07-28 14:25:05.380274272 +0000 UTC m=+136.181151827" Jul 28 14:25:05.692150 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:05.692068 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e21f64-a578-4c74-a7bd-a271d80a92b1" path="/var/lib/kubelet/pods/a2e21f64-a578-4c74-a7bd-a271d80a92b1/volumes" Jul 28 14:25:11.297268 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:11.297235 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f946987f7-jcj4h" Jul 28 14:25:13.398351 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.398322 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c6bb7567-qz6hz"] Jul 28 14:25:13.398753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.398735 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e21f64-a578-4c74-a7bd-a271d80a92b1" containerName="registry" Jul 28 14:25:13.398796 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.398756 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e21f64-a578-4c74-a7bd-a271d80a92b1" containerName="registry" Jul 28 14:25:13.398826 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.398811 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e21f64-a578-4c74-a7bd-a271d80a92b1" containerName="registry" Jul 28 14:25:13.403391 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.402671 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.406362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.406333 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 14:25:13.406591 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.406571 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 14:25:13.407426 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.407394 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xrsm8\"" Jul 28 14:25:13.407919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.407731 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 14:25:13.407919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.407759 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 14:25:13.407919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.407797 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 14:25:13.413224 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.413203 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c6bb7567-qz6hz"] Jul 28 14:25:13.501960 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.501929 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-serving-cert\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.502108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.501974 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-oauth-serving-cert\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.502108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.502073 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-console-config\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.502218 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.502112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-service-ca\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.502218 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.502141 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-oauth-config\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.502218 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.502166 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7mtq\" (UniqueName: \"kubernetes.io/projected/ad66d302-1260-4c80-ab91-5948927819a6-kube-api-access-f7mtq\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.603461 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.603423 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-serving-cert\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.603634 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.603478 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-oauth-serving-cert\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.603634 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.603521 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-console-config\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.603634 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.603546 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-service-ca\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.603634 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.603599 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-oauth-config\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.603634 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.603623 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7mtq\" (UniqueName: \"kubernetes.io/projected/ad66d302-1260-4c80-ab91-5948927819a6-kube-api-access-f7mtq\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.604301 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.604273 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-console-config\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.604408 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.604333 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-service-ca\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.604611 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.604586 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-oauth-serving-cert\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.606308 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.606283 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-serving-cert\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.606529 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.606510 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-oauth-config\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.612822 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.612796 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7mtq\" (UniqueName: \"kubernetes.io/projected/ad66d302-1260-4c80-ab91-5948927819a6-kube-api-access-f7mtq\") pod \"console-77c6bb7567-qz6hz\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.715836 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.715752 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:13.889625 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:13.889596 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c6bb7567-qz6hz"] Jul 28 14:25:13.892465 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:25:13.892435 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad66d302_1260_4c80_ab91_5948927819a6.slice/crio-75183f70b86e8172ad2b86b3035e06cc568cb377eed73efc972d6b983b2c14e8 WatchSource:0}: Error finding container 75183f70b86e8172ad2b86b3035e06cc568cb377eed73efc972d6b983b2c14e8: Status 404 returned error can't find the container with id 75183f70b86e8172ad2b86b3035e06cc568cb377eed73efc972d6b983b2c14e8 Jul 28 14:25:14.326460 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:14.326421 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c6bb7567-qz6hz" event={"ID":"ad66d302-1260-4c80-ab91-5948927819a6","Type":"ContainerStarted","Data":"75183f70b86e8172ad2b86b3035e06cc568cb377eed73efc972d6b983b2c14e8"} Jul 28 14:25:19.982897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:19.982820 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:25:19.983285 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:19.982933 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:25:22.364791 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:22.364408 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-chxst" event={"ID":"4770e716-7472-4083-a75a-35a78ada6fb4","Type":"ContainerStarted","Data":"86245a66370ae2b6d01aada25e1b4416a03cfb8a09a48401e5f03e92df5a82c4"} Jul 28 14:25:22.364791 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:22.364646 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:22.375359 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:22.375318 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-chxst" Jul 28 14:25:22.387168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:22.386735 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-chxst" podStartSLOduration=2.098596539 podStartE2EDuration="20.386715619s" totalImagesPullingTime="18.28811908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:22.38384898 +0000 UTC m=+153.184726532" watchObservedRunningTime="2026-07-28 14:25:22.386715619 +0000 UTC m=+153.187593176" Jul 28 14:25:24.973339 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:24.973309 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f9568f6d5-kclvr"] Jul 28 14:25:24.986102 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:24.986075 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:24.986945 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:24.986909 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f9568f6d5-kclvr"] Jul 28 14:25:24.994968 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:24.994928 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 14:25:25.114449 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114417 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-console-config\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.114647 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114466 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-oauth-serving-cert\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.114647 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114510 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-serving-cert\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.114647 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114535 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-service-ca\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.114825 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114654 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-trusted-ca-bundle\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.114825 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114738 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfdvr\" (UniqueName: \"kubernetes.io/projected/aca62e15-327c-46dd-9379-234e37d591fa-kube-api-access-sfdvr\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.114825 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.114773 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-oauth-config\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215353 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215316 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfdvr\" (UniqueName: \"kubernetes.io/projected/aca62e15-327c-46dd-9379-234e37d591fa-kube-api-access-sfdvr\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215369 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-oauth-config\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215431 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-console-config\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215459 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-oauth-serving-cert\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215482 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-serving-cert\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215503 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-service-ca\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.215536 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.215535 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-trusted-ca-bundle\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.216496 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.216464 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-oauth-serving-cert\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.216496 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.216485 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-console-config\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.217072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.217043 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-service-ca\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.217187 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.217163 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-trusted-ca-bundle\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.218532 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.218471 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-oauth-config\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.218632 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.218606 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-serving-cert\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.224912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.224885 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfdvr\" (UniqueName: \"kubernetes.io/projected/aca62e15-327c-46dd-9379-234e37d591fa-kube-api-access-sfdvr\") pod \"console-7f9568f6d5-kclvr\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.298442 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.298390 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:25.381271 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.381232 2586 generic.go:356] "Generic (PLEG): container finished" podID="b9551376-11dc-4d39-8d3d-ef0abd3d5047" containerID="bd1524948e8a2c090f646a82d849748a63790909646e7c971847e9298cafd81d" exitCode=0 Jul 28 14:25:25.381451 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.381304 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" event={"ID":"b9551376-11dc-4d39-8d3d-ef0abd3d5047","Type":"ContainerDied","Data":"bd1524948e8a2c090f646a82d849748a63790909646e7c971847e9298cafd81d"} Jul 28 14:25:25.381954 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.381729 2586 scope.go:117] "RemoveContainer" containerID="bd1524948e8a2c090f646a82d849748a63790909646e7c971847e9298cafd81d" Jul 28 14:25:25.561919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:25.561809 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f9568f6d5-kclvr"] Jul 28 14:25:25.565285 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:25:25.565254 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca62e15_327c_46dd_9379_234e37d591fa.slice/crio-33dba662fd355339994c54e54f7631227203c902fc29dae3715b633e995ce240 WatchSource:0}: Error finding container 33dba662fd355339994c54e54f7631227203c902fc29dae3715b633e995ce240: Status 404 returned error can't find the container with id 33dba662fd355339994c54e54f7631227203c902fc29dae3715b633e995ce240 Jul 28 14:25:26.387961 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:26.387925 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss" event={"ID":"b9551376-11dc-4d39-8d3d-ef0abd3d5047","Type":"ContainerStarted","Data":"d50eb0d147445fad61f036378936f85380386a90c4484d73907d688bb9984964"} Jul 28 14:25:26.390095 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:26.390067 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c6bb7567-qz6hz" event={"ID":"ad66d302-1260-4c80-ab91-5948927819a6","Type":"ContainerStarted","Data":"2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177"} Jul 28 14:25:26.391877 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:26.391849 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9568f6d5-kclvr" event={"ID":"aca62e15-327c-46dd-9379-234e37d591fa","Type":"ContainerStarted","Data":"40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9"} Jul 28 14:25:26.392009 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:26.391899 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9568f6d5-kclvr" event={"ID":"aca62e15-327c-46dd-9379-234e37d591fa","Type":"ContainerStarted","Data":"33dba662fd355339994c54e54f7631227203c902fc29dae3715b633e995ce240"} Jul 28 14:25:26.438126 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:26.438064 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77c6bb7567-qz6hz" podStartSLOduration=1.94519487 podStartE2EDuration="13.438042171s" totalImagesPullingTime="11.492847301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:26.434191458 +0000 UTC m=+157.235069015" watchObservedRunningTime="2026-07-28 14:25:26.438042171 +0000 UTC m=+157.238919719" Jul 28 14:25:26.454491 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:26.454445 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f9568f6d5-kclvr" podStartSLOduration=2.454426019 podStartE2EDuration="2.454426019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:25:26.45396686 +0000 UTC m=+157.254844417" watchObservedRunningTime="2026-07-28 14:25:26.454426019 +0000 UTC m=+157.255303578" Jul 28 14:25:33.716758 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:33.716726 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:33.717188 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:33.717027 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:33.721385 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:33.721365 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:34.434953 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:34.434919 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:25:35.298662 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:35.298628 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:35.299043 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:35.298708 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:35.303309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:35.303288 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:35.439778 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:35.439747 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:25:35.492530 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:35.492501 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c6bb7567-qz6hz"] Jul 28 14:25:39.988207 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:39.988177 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:25:39.991808 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:39.991785 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7469fb649c-dhfdp" Jul 28 14:25:40.456426 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:40.456394 2586 generic.go:356] "Generic (PLEG): container finished" podID="4df310a6-b860-4f23-9e5e-e4da2c5c3618" containerID="cfca582538c0a2f3654762e592a1ca83104fe93eadd56d45ddd2b80b13ed861b" exitCode=0 Jul 28 14:25:40.456682 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:40.456472 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" event={"ID":"4df310a6-b860-4f23-9e5e-e4da2c5c3618","Type":"ContainerDied","Data":"cfca582538c0a2f3654762e592a1ca83104fe93eadd56d45ddd2b80b13ed861b"} Jul 28 14:25:40.456880 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:40.456865 2586 scope.go:117] "RemoveContainer" containerID="cfca582538c0a2f3654762e592a1ca83104fe93eadd56d45ddd2b80b13ed861b" Jul 28 14:25:41.462908 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:25:41.462825 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c2jrg" event={"ID":"4df310a6-b860-4f23-9e5e-e4da2c5c3618","Type":"ContainerStarted","Data":"84847be57f838ef024943e591c4ab3106a88bfc52b2c312243f90e08d7a274f1"} Jul 28 14:26:02.460645 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.460597 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77c6bb7567-qz6hz" podUID="ad66d302-1260-4c80-ab91-5948927819a6" containerName="console" containerID="cri-o://2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177" gracePeriod=15 Jul 28 14:26:02.731391 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.731335 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:26:02.841031 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841005 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7mtq\" (UniqueName: \"kubernetes.io/projected/ad66d302-1260-4c80-ab91-5948927819a6-kube-api-access-f7mtq\") pod \"ad66d302-1260-4c80-ab91-5948927819a6\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " Jul 28 14:26:02.841031 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841034 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-console-config\") pod \"ad66d302-1260-4c80-ab91-5948927819a6\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " Jul 28 14:26:02.841212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841064 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-serving-cert\") pod \"ad66d302-1260-4c80-ab91-5948927819a6\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " Jul 28 14:26:02.841212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841082 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-service-ca\") pod \"ad66d302-1260-4c80-ab91-5948927819a6\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " Jul 28 14:26:02.841212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841097 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-oauth-serving-cert\") pod \"ad66d302-1260-4c80-ab91-5948927819a6\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " Jul 28 14:26:02.841212 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841170 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-oauth-config\") pod \"ad66d302-1260-4c80-ab91-5948927819a6\" (UID: \"ad66d302-1260-4c80-ab91-5948927819a6\") " Jul 28 14:26:02.841479 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841450 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-console-config" (OuterVolumeSpecName: "console-config") pod "ad66d302-1260-4c80-ab91-5948927819a6" (UID: "ad66d302-1260-4c80-ab91-5948927819a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:26:02.841567 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841478 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "ad66d302-1260-4c80-ab91-5948927819a6" (UID: "ad66d302-1260-4c80-ab91-5948927819a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:26:02.841631 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.841577 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ad66d302-1260-4c80-ab91-5948927819a6" (UID: "ad66d302-1260-4c80-ab91-5948927819a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:26:02.843380 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.843359 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ad66d302-1260-4c80-ab91-5948927819a6" (UID: "ad66d302-1260-4c80-ab91-5948927819a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:02.843485 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.843378 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ad66d302-1260-4c80-ab91-5948927819a6" (UID: "ad66d302-1260-4c80-ab91-5948927819a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:02.843485 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.843397 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad66d302-1260-4c80-ab91-5948927819a6-kube-api-access-f7mtq" (OuterVolumeSpecName: "kube-api-access-f7mtq") pod "ad66d302-1260-4c80-ab91-5948927819a6" (UID: "ad66d302-1260-4c80-ab91-5948927819a6"). InnerVolumeSpecName "kube-api-access-f7mtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:26:02.941925 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.941904 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-oauth-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:02.941925 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.941924 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7mtq\" (UniqueName: \"kubernetes.io/projected/ad66d302-1260-4c80-ab91-5948927819a6-kube-api-access-f7mtq\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:02.942053 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.941934 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-console-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:02.942053 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.941943 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ad66d302-1260-4c80-ab91-5948927819a6-console-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:02.942053 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.941951 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:02.942053 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:02.941960 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ad66d302-1260-4c80-ab91-5948927819a6-oauth-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:03.555682 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.555649 2586 generic.go:356] "Generic (PLEG): container finished" podID="ad66d302-1260-4c80-ab91-5948927819a6" containerID="2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177" exitCode=0 Jul 28 14:26:03.556080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.555696 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c6bb7567-qz6hz" Jul 28 14:26:03.556080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.555743 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c6bb7567-qz6hz" event={"ID":"ad66d302-1260-4c80-ab91-5948927819a6","Type":"ContainerDied","Data":"2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177"} Jul 28 14:26:03.556080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.555794 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c6bb7567-qz6hz" event={"ID":"ad66d302-1260-4c80-ab91-5948927819a6","Type":"ContainerDied","Data":"75183f70b86e8172ad2b86b3035e06cc568cb377eed73efc972d6b983b2c14e8"} Jul 28 14:26:03.556080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.555816 2586 scope.go:117] "RemoveContainer" containerID="2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177" Jul 28 14:26:03.564429 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.564411 2586 scope.go:117] "RemoveContainer" containerID="2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177" Jul 28 14:26:03.564678 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:03.564660 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177\": container with ID starting with 2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177 not found: ID does not exist" containerID="2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177" Jul 28 14:26:03.564729 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.564686 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177"} err="failed to get container status \"2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177\": rpc error: code = NotFound desc = could not find container \"2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177\": container with ID starting with 2ed2ab3e4b30d7560fec1a730ab396eabe3a6aaadc2eec3d3fb7fea100155177 not found: ID does not exist" Jul 28 14:26:03.577096 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.577076 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c6bb7567-qz6hz"] Jul 28 14:26:03.581311 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.581293 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77c6bb7567-qz6hz"] Jul 28 14:26:03.689562 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:03.689532 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad66d302-1260-4c80-ab91-5948927819a6" path="/var/lib/kubelet/pods/ad66d302-1260-4c80-ab91-5948927819a6/volumes" Jul 28 14:26:15.634713 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.634673 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:26:15.635211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.635001 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="alertmanager" containerID="cri-o://02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da" gracePeriod=120 Jul 28 14:26:15.635211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.635065 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-metric" containerID="cri-o://43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3" gracePeriod=120 Jul 28 14:26:15.635211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.635090 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-web" containerID="cri-o://b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24" gracePeriod=120 Jul 28 14:26:15.635211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.635110 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="config-reloader" containerID="cri-o://a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18" gracePeriod=120 Jul 28 14:26:15.635211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.635165 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="prom-label-proxy" containerID="cri-o://a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8" gracePeriod=120 Jul 28 14:26:15.635211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:15.635150 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy" containerID="cri-o://6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497" gracePeriod=120 Jul 28 14:26:16.610908 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.610872 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8" exitCode=0 Jul 28 14:26:16.610908 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.610897 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497" exitCode=0 Jul 28 14:26:16.610908 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.610906 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18" exitCode=0 Jul 28 14:26:16.610908 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.610914 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da" exitCode=0 Jul 28 14:26:16.611176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.610950 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8"} Jul 28 14:26:16.611176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.611031 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497"} Jul 28 14:26:16.611176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.611075 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18"} Jul 28 14:26:16.611176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.611117 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da"} Jul 28 14:26:16.870258 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:16.870211 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.053078 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053051 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053078 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053081 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-config-out\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053102 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-web\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053119 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-metrics-client-ca\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053143 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053164 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-292lg\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-kube-api-access-292lg\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053275 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-tls-assets\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053310 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-main-db\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053381 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-main-tls\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053430 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-web-config\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053477 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053504 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-config-volume\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053527 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:26:17.053584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053536 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-cluster-tls-config\") pod \"f4d59513-da52-4f5a-8f96-00528f860f26\" (UID: \"f4d59513-da52-4f5a-8f96-00528f860f26\") " Jul 28 14:26:17.053912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053649 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:26:17.053912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053855 2586 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-metrics-client-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.053912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.053874 2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.054208 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.054170 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:26:17.056494 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.056457 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-kube-api-access-292lg" (OuterVolumeSpecName: "kube-api-access-292lg") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "kube-api-access-292lg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:26:17.057255 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057206 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:26:17.057255 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057207 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.057255 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057243 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.057569 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057526 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.057569 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057535 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-config-out" (OuterVolumeSpecName: "config-out") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:26:17.057714 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057643 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-config-volume" (OuterVolumeSpecName: "config-volume") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.057714 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.057701 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.060364 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.060345 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.066659 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.066634 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-web-config" (OuterVolumeSpecName: "web-config") pod "f4d59513-da52-4f5a-8f96-00528f860f26" (UID: "f4d59513-da52-4f5a-8f96-00528f860f26"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155135 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-292lg\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-kube-api-access-292lg\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155171 2586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4d59513-da52-4f5a-8f96-00528f860f26-tls-assets\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155182 2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-alertmanager-main-db\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155192 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-main-tls\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155201 2586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-web-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155210 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155220 2586 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-config-volume\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155214 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155228 2586 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-cluster-tls-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155579 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155236 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155579 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155244 2586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4d59513-da52-4f5a-8f96-00528f860f26-config-out\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.155579 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.155252 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4d59513-da52-4f5a-8f96-00528f860f26-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:26:17.617938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.617908 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3" exitCode=0 Jul 28 14:26:17.617938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.617932 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4d59513-da52-4f5a-8f96-00528f860f26" containerID="b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24" exitCode=0 Jul 28 14:26:17.618131 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.617991 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.618131 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.617990 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3"} Jul 28 14:26:17.618131 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.618078 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24"} Jul 28 14:26:17.618131 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.618124 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f4d59513-da52-4f5a-8f96-00528f860f26","Type":"ContainerDied","Data":"9359302d849b549598bf9c38bc521099616c0105b7d5e344690f799e01ae9ee1"} Jul 28 14:26:17.618314 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.618158 2586 scope.go:117] "RemoveContainer" containerID="a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8" Jul 28 14:26:17.626086 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.626066 2586 scope.go:117] "RemoveContainer" containerID="43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3" Jul 28 14:26:17.633086 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.633069 2586 scope.go:117] "RemoveContainer" containerID="6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497" Jul 28 14:26:17.639607 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.639586 2586 scope.go:117] "RemoveContainer" containerID="b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24" Jul 28 14:26:17.642117 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.642071 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:26:17.646580 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.646536 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:26:17.646740 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.646727 2586 scope.go:117] "RemoveContainer" containerID="a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18" Jul 28 14:26:17.653506 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.653490 2586 scope.go:117] "RemoveContainer" containerID="02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da" Jul 28 14:26:17.660147 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.660126 2586 scope.go:117] "RemoveContainer" containerID="95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344" Jul 28 14:26:17.666921 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.666907 2586 scope.go:117] "RemoveContainer" containerID="a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8" Jul 28 14:26:17.667146 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.667129 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8\": container with ID starting with a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8 not found: ID does not exist" containerID="a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8" Jul 28 14:26:17.667202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667154 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8"} err="failed to get container status \"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8\": rpc error: code = NotFound desc = could not find container \"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8\": container with ID starting with a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8 not found: ID does not exist" Jul 28 14:26:17.667202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667175 2586 scope.go:117] "RemoveContainer" containerID="43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3" Jul 28 14:26:17.667405 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.667390 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3\": container with ID starting with 43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3 not found: ID does not exist" containerID="43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3" Jul 28 14:26:17.667444 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667410 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3"} err="failed to get container status \"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3\": rpc error: code = NotFound desc = could not find container \"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3\": container with ID starting with 43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3 not found: ID does not exist" Jul 28 14:26:17.667444 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667426 2586 scope.go:117] "RemoveContainer" containerID="6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497" Jul 28 14:26:17.667669 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.667654 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497\": container with ID starting with 6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497 not found: ID does not exist" containerID="6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497" Jul 28 14:26:17.667715 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667673 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497"} err="failed to get container status \"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497\": rpc error: code = NotFound desc = could not find container \"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497\": container with ID starting with 6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497 not found: ID does not exist" Jul 28 14:26:17.667715 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667684 2586 scope.go:117] "RemoveContainer" containerID="b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24" Jul 28 14:26:17.667917 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.667901 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24\": container with ID starting with b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24 not found: ID does not exist" containerID="b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24" Jul 28 14:26:17.667961 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667918 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24"} err="failed to get container status \"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24\": rpc error: code = NotFound desc = could not find container \"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24\": container with ID starting with b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24 not found: ID does not exist" Jul 28 14:26:17.667961 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.667931 2586 scope.go:117] "RemoveContainer" containerID="a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18" Jul 28 14:26:17.668109 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.668094 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18\": container with ID starting with a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18 not found: ID does not exist" containerID="a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18" Jul 28 14:26:17.668150 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668112 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18"} err="failed to get container status \"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18\": rpc error: code = NotFound desc = could not find container \"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18\": container with ID starting with a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18 not found: ID does not exist" Jul 28 14:26:17.668150 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668124 2586 scope.go:117] "RemoveContainer" containerID="02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da" Jul 28 14:26:17.668348 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.668332 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da\": container with ID starting with 02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da not found: ID does not exist" containerID="02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da" Jul 28 14:26:17.668399 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668354 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da"} err="failed to get container status \"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da\": rpc error: code = NotFound desc = could not find container \"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da\": container with ID starting with 02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da not found: ID does not exist" Jul 28 14:26:17.668399 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668369 2586 scope.go:117] "RemoveContainer" containerID="95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344" Jul 28 14:26:17.668623 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:26:17.668601 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344\": container with ID starting with 95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344 not found: ID does not exist" containerID="95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344" Jul 28 14:26:17.668675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668621 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344"} err="failed to get container status \"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344\": rpc error: code = NotFound desc = could not find container \"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344\": container with ID starting with 95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344 not found: ID does not exist" Jul 28 14:26:17.668675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668635 2586 scope.go:117] "RemoveContainer" containerID="a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8" Jul 28 14:26:17.668849 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668833 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8"} err="failed to get container status \"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8\": rpc error: code = NotFound desc = could not find container \"a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8\": container with ID starting with a458b7b472bd9d92ada0ecb766e8adb9eb96cc74ffd96d673819a20b0b2580b8 not found: ID does not exist" Jul 28 14:26:17.668889 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.668849 2586 scope.go:117] "RemoveContainer" containerID="43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3" Jul 28 14:26:17.669071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669052 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3"} err="failed to get container status \"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3\": rpc error: code = NotFound desc = could not find container \"43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3\": container with ID starting with 43deff4039ebb2455a20c6b6982553405395a6d2b6a333a8a84dfb14b00e06d3 not found: ID does not exist" Jul 28 14:26:17.669118 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669073 2586 scope.go:117] "RemoveContainer" containerID="6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497" Jul 28 14:26:17.669275 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669256 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497"} err="failed to get container status \"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497\": rpc error: code = NotFound desc = could not find container \"6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497\": container with ID starting with 6d4f6a9ddfe28c34612d69fc8b384127c3e507279241c46fe8cfac6318fd8497 not found: ID does not exist" Jul 28 14:26:17.669275 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669273 2586 scope.go:117] "RemoveContainer" containerID="b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24" Jul 28 14:26:17.669469 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669454 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24"} err="failed to get container status \"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24\": rpc error: code = NotFound desc = could not find container \"b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24\": container with ID starting with b9714b84e70f0e82d70f29605d217adaa16f989118d4d52c15e330f76ec97f24 not found: ID does not exist" Jul 28 14:26:17.669509 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669469 2586 scope.go:117] "RemoveContainer" containerID="a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18" Jul 28 14:26:17.669745 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669726 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18"} err="failed to get container status \"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18\": rpc error: code = NotFound desc = could not find container \"a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18\": container with ID starting with a369fd837f045a056f74074179ac90e83bba67c5085bf1b29cd53e320f901c18 not found: ID does not exist" Jul 28 14:26:17.669745 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669744 2586 scope.go:117] "RemoveContainer" containerID="02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da" Jul 28 14:26:17.669938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669923 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da"} err="failed to get container status \"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da\": rpc error: code = NotFound desc = could not find container \"02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da\": container with ID starting with 02aff10f94ba872ce0f3e986ecbb98c48393dfb0b541e5306a65ad639d2d94da not found: ID does not exist" Jul 28 14:26:17.669980 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.669938 2586 scope.go:117] "RemoveContainer" containerID="95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344" Jul 28 14:26:17.670153 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.670138 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344"} err="failed to get container status \"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344\": rpc error: code = NotFound desc = could not find container \"95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344\": container with ID starting with 95ab87e59fa995ace401ae4f22339e9d8867dfc40089cd2a8e5c2ec76e289344 not found: ID does not exist" Jul 28 14:26:17.673509 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673489 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:26:17.673813 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673799 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="config-reloader" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673815 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="config-reloader" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673826 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad66d302-1260-4c80-ab91-5948927819a6" containerName="console" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673832 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad66d302-1260-4c80-ab91-5948927819a6" containerName="console" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673841 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-metric" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673846 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-metric" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673854 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy" Jul 28 14:26:17.673860 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673859 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673870 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-web" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673876 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-web" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673883 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="prom-label-proxy" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673888 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="prom-label-proxy" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673897 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="init-config-reloader" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673924 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="init-config-reloader" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673933 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="alertmanager" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673939 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="alertmanager" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673982 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="config-reloader" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673992 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-metric" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.673998 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad66d302-1260-4c80-ab91-5948927819a6" containerName="console" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.674005 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.674011 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="kube-rbac-proxy-web" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.674016 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="prom-label-proxy" Jul 28 14:26:17.674071 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.674022 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" containerName="alertmanager" Jul 28 14:26:17.677728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.677714 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.680697 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680525 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-wgpzt\"" Jul 28 14:26:17.680697 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680562 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 14:26:17.680697 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680581 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 14:26:17.680697 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680587 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 14:26:17.680697 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680580 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 14:26:17.680697 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680666 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 14:26:17.681005 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680752 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 14:26:17.681005 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680946 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 14:26:17.681005 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.680947 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 14:26:17.690194 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.690172 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 14:26:17.693711 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.693686 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d59513-da52-4f5a-8f96-00528f860f26" path="/var/lib/kubelet/pods/f4d59513-da52-4f5a-8f96-00528f860f26/volumes" Jul 28 14:26:17.694199 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.694184 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:26:17.862081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862049 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862082 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862103 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-config-out\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862129 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862181 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj2jm\" (UniqueName: \"kubernetes.io/projected/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-kube-api-access-zj2jm\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862215 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862240 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862256 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862256 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862312 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862355 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-web-config\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862421 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-config-volume\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862452 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.862498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.862477 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963206 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963241 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963258 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963374 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963409 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-web-config\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963462 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-config-volume\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963592 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963653 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963687 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963715 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963741 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-config-out\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.963776 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963775 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.964203 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963786 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.964203 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.963837 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj2jm\" (UniqueName: \"kubernetes.io/projected/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-kube-api-access-zj2jm\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.964203 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.964161 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.964893 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.964867 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.966070 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.966044 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-tls-assets\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.966806 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.966771 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.966900 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.966885 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-config-out\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.966956 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.966911 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.967010 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.966944 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.967420 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.967397 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-web-config\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.967496 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.967475 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.967713 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.967694 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.967757 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.967694 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-config-volume\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.972072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.972053 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj2jm\" (UniqueName: \"kubernetes.io/projected/70f08444-cbcc-4825-8a53-6c4fe6f2ad37-kube-api-access-zj2jm\") pod \"alertmanager-main-0\" (UID: \"70f08444-cbcc-4825-8a53-6c4fe6f2ad37\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:17.989389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:17.989370 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 14:26:18.123461 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:18.123433 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 14:26:18.126670 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:26:18.126642 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f08444_cbcc_4825_8a53_6c4fe6f2ad37.slice/crio-38833ae80305aca7143327bfb23c278e8b78459975961ef286d223aef0037ad4 WatchSource:0}: Error finding container 38833ae80305aca7143327bfb23c278e8b78459975961ef286d223aef0037ad4: Status 404 returned error can't find the container with id 38833ae80305aca7143327bfb23c278e8b78459975961ef286d223aef0037ad4 Jul 28 14:26:18.623241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:18.623207 2586 generic.go:356] "Generic (PLEG): container finished" podID="70f08444-cbcc-4825-8a53-6c4fe6f2ad37" containerID="2a4808934978cf25a2aa35165bf5fcc65fb68444048fb8ee448b15be342af04b" exitCode=0 Jul 28 14:26:18.623403 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:18.623291 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerDied","Data":"2a4808934978cf25a2aa35165bf5fcc65fb68444048fb8ee448b15be342af04b"} Jul 28 14:26:18.623403 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:18.623378 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"38833ae80305aca7143327bfb23c278e8b78459975961ef286d223aef0037ad4"} Jul 28 14:26:19.630932 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.630901 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"9e45f8c194554dcfff7ead03566fe1dddfb51e1e058e96c7a37420090a48f576"} Jul 28 14:26:19.631263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.630962 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"73b65cb70e22f0c4cffcafed96d40e63914d861a3466c23c159b86c57bf7ad1c"} Jul 28 14:26:19.631263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.630989 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"2059add6f28c645ce4caf3c6d5eea9c01c993f31fee0c8c6fd9d096d685d766e"} Jul 28 14:26:19.631263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.631013 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"144f81a8fc458410165561e0a86f34c5f8fc255c4ade3c91fd7e3b8985630e8c"} Jul 28 14:26:19.631263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.631036 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"73701e73ff96f1dc1f5cb4577f7601ec3c70b09b2f0329db8f732ab9bd6bd892"} Jul 28 14:26:19.631263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.631059 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"70f08444-cbcc-4825-8a53-6c4fe6f2ad37","Type":"ContainerStarted","Data":"f7f71763609221f57a8900d7a1a693390f570318723481f71a0525ff3a5e43a0"} Jul 28 14:26:19.659580 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.659540 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s"] Jul 28 14:26:19.662061 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.662042 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.663998 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.663961 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.6470363839999997 podStartE2EDuration="2.663947719s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.911335ms" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:19.661437119 +0000 UTC m=+210.462314676" watchObservedRunningTime="2026-07-28 14:26:19.663947719 +0000 UTC m=+210.464825275" Jul 28 14:26:19.665480 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.665460 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 28 14:26:19.665616 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.665491 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 28 14:26:19.665780 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.665764 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 28 14:26:19.665832 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.665770 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c2cj7\"" Jul 28 14:26:19.665832 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.665809 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 28 14:26:19.666713 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.666696 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 28 14:26:19.672573 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.671688 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 28 14:26:19.691913 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.691883 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s"] Jul 28 14:26:19.780459 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.780430 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-metrics-client-ca\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.780615 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.780471 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-telemeter-client-tls\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.780615 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.780527 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-serving-certs-ca-bundle\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.780756 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.780738 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-secret-telemeter-client\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.780886 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.780866 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.780946 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.780922 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.781176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.781099 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-federate-client-tls\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.781176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.781137 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fh68\" (UniqueName: \"kubernetes.io/projected/8eadbf34-39b7-4208-9efd-1b8f3b00129b-kube-api-access-2fh68\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882181 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882106 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-federate-client-tls\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882181 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882142 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fh68\" (UniqueName: \"kubernetes.io/projected/8eadbf34-39b7-4208-9efd-1b8f3b00129b-kube-api-access-2fh68\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882416 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882180 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-metrics-client-ca\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882416 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882210 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-telemeter-client-tls\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882416 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882242 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-serving-certs-ca-bundle\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882416 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882266 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-secret-telemeter-client\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882660 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882423 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.882660 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882493 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.883020 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882993 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-metrics-client-ca\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.883117 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.882993 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-serving-certs-ca-bundle\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.883537 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.883511 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8eadbf34-39b7-4208-9efd-1b8f3b00129b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.884753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.884732 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-federate-client-tls\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.884840 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.884773 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-telemeter-client-tls\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.885064 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.885046 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.885166 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.885148 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8eadbf34-39b7-4208-9efd-1b8f3b00129b-secret-telemeter-client\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.890763 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.890743 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fh68\" (UniqueName: \"kubernetes.io/projected/8eadbf34-39b7-4208-9efd-1b8f3b00129b-kube-api-access-2fh68\") pod \"telemeter-client-d7fbdd4f8-vss6s\" (UID: \"8eadbf34-39b7-4208-9efd-1b8f3b00129b\") " pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:19.977765 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:19.977729 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" Jul 28 14:26:20.108859 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:20.108836 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s"] Jul 28 14:26:20.111396 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:26:20.111368 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eadbf34_39b7_4208_9efd_1b8f3b00129b.slice/crio-ae16be26abe6cb8d07d7dc2edbcff61dfda97591a622ada642c63d3d4c8ecb94 WatchSource:0}: Error finding container ae16be26abe6cb8d07d7dc2edbcff61dfda97591a622ada642c63d3d4c8ecb94: Status 404 returned error can't find the container with id ae16be26abe6cb8d07d7dc2edbcff61dfda97591a622ada642c63d3d4c8ecb94 Jul 28 14:26:20.636021 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:20.635991 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" event={"ID":"8eadbf34-39b7-4208-9efd-1b8f3b00129b","Type":"ContainerStarted","Data":"ae16be26abe6cb8d07d7dc2edbcff61dfda97591a622ada642c63d3d4c8ecb94"} Jul 28 14:26:22.652347 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:22.652310 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" event={"ID":"8eadbf34-39b7-4208-9efd-1b8f3b00129b","Type":"ContainerStarted","Data":"784582fd671263d199ac6606ef88933c306187dbdba13d0fac4d5ec03a937943"} Jul 28 14:26:22.652729 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:22.652388 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" event={"ID":"8eadbf34-39b7-4208-9efd-1b8f3b00129b","Type":"ContainerStarted","Data":"929c5df5816014db6b9134fa8069af11e9738e6580cccf86234b2e9e3521f8a7"} Jul 28 14:26:22.652729 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:22.652408 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" event={"ID":"8eadbf34-39b7-4208-9efd-1b8f3b00129b","Type":"ContainerStarted","Data":"d88e5f39d1b197522ade67305fa2561ee8a206c3359b5e47e23529cd7421cc7e"} Jul 28 14:26:22.678250 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:22.678205 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-d7fbdd4f8-vss6s" podStartSLOduration=1.992207697 podStartE2EDuration="3.678189832s" totalImagesPullingTime="1.685982135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:22.67600863 +0000 UTC m=+213.476886187" watchObservedRunningTime="2026-07-28 14:26:22.678189832 +0000 UTC m=+213.479067389" Jul 28 14:26:23.495108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.495076 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8d87fc8d6-fhxtq"] Jul 28 14:26:23.497400 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.497386 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.508356 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.508334 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d87fc8d6-fhxtq"] Jul 28 14:26:23.510803 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.510780 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-oauth-serving-cert\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.510914 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.510835 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-console-config\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.510914 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.510881 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-serving-cert\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.511036 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.510934 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-service-ca\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.511036 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.510957 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgj9f\" (UniqueName: \"kubernetes.io/projected/291758e4-a31f-4d4e-8c24-46306a15afee-kube-api-access-dgj9f\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.511036 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.511004 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-oauth-config\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.511184 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.511088 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-trusted-ca-bundle\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612324 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612287 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-console-config\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612324 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612332 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-serving-cert\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612361 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-service-ca\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612388 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgj9f\" (UniqueName: \"kubernetes.io/projected/291758e4-a31f-4d4e-8c24-46306a15afee-kube-api-access-dgj9f\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612411 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-oauth-config\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612436 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-trusted-ca-bundle\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.612501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.612458 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-oauth-serving-cert\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.613126 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.613106 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-service-ca\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.613206 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.613103 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-console-config\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.613274 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.613251 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-oauth-serving-cert\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.613309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.613292 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-trusted-ca-bundle\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.614782 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.614759 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-oauth-config\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.614883 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.614856 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-serving-cert\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.621766 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.621744 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgj9f\" (UniqueName: \"kubernetes.io/projected/291758e4-a31f-4d4e-8c24-46306a15afee-kube-api-access-dgj9f\") pod \"console-8d87fc8d6-fhxtq\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.806356 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.806327 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:23.955973 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:23.955944 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d87fc8d6-fhxtq"] Jul 28 14:26:23.959004 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:26:23.958972 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291758e4_a31f_4d4e_8c24_46306a15afee.slice/crio-890f53463097020f83a8abc30042f1f0bd7edf1392d8ae97d182925511386c60 WatchSource:0}: Error finding container 890f53463097020f83a8abc30042f1f0bd7edf1392d8ae97d182925511386c60: Status 404 returned error can't find the container with id 890f53463097020f83a8abc30042f1f0bd7edf1392d8ae97d182925511386c60 Jul 28 14:26:24.662956 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:24.662923 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d87fc8d6-fhxtq" event={"ID":"291758e4-a31f-4d4e-8c24-46306a15afee","Type":"ContainerStarted","Data":"76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf"} Jul 28 14:26:24.663126 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:24.662968 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d87fc8d6-fhxtq" event={"ID":"291758e4-a31f-4d4e-8c24-46306a15afee","Type":"ContainerStarted","Data":"890f53463097020f83a8abc30042f1f0bd7edf1392d8ae97d182925511386c60"} Jul 28 14:26:24.681987 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:24.681944 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8d87fc8d6-fhxtq" podStartSLOduration=1.6819305469999999 podStartE2EDuration="1.681930547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:26:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:26:24.680737106 +0000 UTC m=+215.481614663" watchObservedRunningTime="2026-07-28 14:26:24.681930547 +0000 UTC m=+215.482808103" Jul 28 14:26:33.806831 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:33.806801 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:33.807257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:33.806852 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:33.811417 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:33.811394 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:34.715852 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:34.715825 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:26:34.780537 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:34.780510 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f9568f6d5-kclvr"] Jul 28 14:26:59.799605 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:26:59.799530 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f9568f6d5-kclvr" podUID="aca62e15-327c-46dd-9379-234e37d591fa" containerName="console" containerID="cri-o://40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9" gracePeriod=15 Jul 28 14:27:00.042382 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.042351 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:27:00.116943 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.116914 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-serving-cert\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117128 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.116979 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfdvr\" (UniqueName: \"kubernetes.io/projected/aca62e15-327c-46dd-9379-234e37d591fa-kube-api-access-sfdvr\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117128 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.116997 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-oauth-config\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117128 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117015 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-oauth-serving-cert\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117128 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117031 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-console-config\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117128 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117056 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-service-ca\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117128 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117079 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-trusted-ca-bundle\") pod \"aca62e15-327c-46dd-9379-234e37d591fa\" (UID: \"aca62e15-327c-46dd-9379-234e37d591fa\") " Jul 28 14:27:00.117624 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117591 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-service-ca" (OuterVolumeSpecName: "service-ca") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:27:00.117624 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117603 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:27:00.117624 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117611 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-console-config" (OuterVolumeSpecName: "console-config") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:27:00.117844 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.117661 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:27:00.119195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.119173 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:27:00.119282 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.119175 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aca62e15-327c-46dd-9379-234e37d591fa-kube-api-access-sfdvr" (OuterVolumeSpecName: "kube-api-access-sfdvr") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "kube-api-access-sfdvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:27:00.119282 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.119214 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aca62e15-327c-46dd-9379-234e37d591fa" (UID: "aca62e15-327c-46dd-9379-234e37d591fa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:27:00.218276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218232 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfdvr\" (UniqueName: \"kubernetes.io/projected/aca62e15-327c-46dd-9379-234e37d591fa-kube-api-access-sfdvr\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.218276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218270 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-oauth-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.218276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218284 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-oauth-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.218515 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218297 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-console-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.218515 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218310 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.218515 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218320 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca62e15-327c-46dd-9379-234e37d591fa-trusted-ca-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.218515 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.218333 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aca62e15-327c-46dd-9379-234e37d591fa-console-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:27:00.813279 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.813238 2586 generic.go:356] "Generic (PLEG): container finished" podID="aca62e15-327c-46dd-9379-234e37d591fa" containerID="40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9" exitCode=0 Jul 28 14:27:00.813279 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.813277 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9568f6d5-kclvr" event={"ID":"aca62e15-327c-46dd-9379-234e37d591fa","Type":"ContainerDied","Data":"40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9"} Jul 28 14:27:00.813839 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.813331 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9568f6d5-kclvr" event={"ID":"aca62e15-327c-46dd-9379-234e37d591fa","Type":"ContainerDied","Data":"33dba662fd355339994c54e54f7631227203c902fc29dae3715b633e995ce240"} Jul 28 14:27:00.813839 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.813286 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9568f6d5-kclvr" Jul 28 14:27:00.813839 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.813350 2586 scope.go:117] "RemoveContainer" containerID="40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9" Jul 28 14:27:00.821843 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.821829 2586 scope.go:117] "RemoveContainer" containerID="40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9" Jul 28 14:27:00.822083 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:27:00.822066 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9\": container with ID starting with 40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9 not found: ID does not exist" containerID="40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9" Jul 28 14:27:00.822125 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.822090 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9"} err="failed to get container status \"40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9\": rpc error: code = NotFound desc = could not find container \"40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9\": container with ID starting with 40f72b8b5e4fd6764865ef55848a3ce6a3497779c0c2ad95691d833b93b1a0e9 not found: ID does not exist" Jul 28 14:27:00.834110 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.834084 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f9568f6d5-kclvr"] Jul 28 14:27:00.836514 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:00.836495 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f9568f6d5-kclvr"] Jul 28 14:27:01.689950 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:01.689905 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aca62e15-327c-46dd-9379-234e37d591fa" path="/var/lib/kubelet/pods/aca62e15-327c-46dd-9379-234e37d591fa/volumes" Jul 28 14:27:49.588180 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:27:49.588149 2586 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 14:28:02.676963 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.676929 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b85f648d5-hcrv5"] Jul 28 14:28:02.679190 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.677251 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aca62e15-327c-46dd-9379-234e37d591fa" containerName="console" Jul 28 14:28:02.679190 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.677263 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="aca62e15-327c-46dd-9379-234e37d591fa" containerName="console" Jul 28 14:28:02.679190 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.677339 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="aca62e15-327c-46dd-9379-234e37d591fa" containerName="console" Jul 28 14:28:02.679994 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.679979 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.692480 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.692458 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b85f648d5-hcrv5"] Jul 28 14:28:02.823614 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823588 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-oauth-serving-cert\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.823743 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823618 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-oauth-config\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.823743 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823636 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-service-ca\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.823743 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823662 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cvb7\" (UniqueName: \"kubernetes.io/projected/6d470205-91ef-49c4-a40e-fc5a2d131ddb-kube-api-access-8cvb7\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.823743 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823717 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-config\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.823879 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823784 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-trusted-ca-bundle\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.823879 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.823812 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-serving-cert\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.924980 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.924958 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-oauth-serving-cert\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.924986 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-oauth-config\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925001 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-service-ca\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925080 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925023 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cvb7\" (UniqueName: \"kubernetes.io/projected/6d470205-91ef-49c4-a40e-fc5a2d131ddb-kube-api-access-8cvb7\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925332 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925307 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-config\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925378 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925361 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-trusted-ca-bundle\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925427 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925408 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-serving-cert\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925718 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925697 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-service-ca\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.925840 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.925797 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-oauth-serving-cert\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.926093 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.926073 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-trusted-ca-bundle\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.926144 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.926074 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-config\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.927737 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.927693 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-oauth-config\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.927737 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.927715 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-serving-cert\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.932834 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.932812 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cvb7\" (UniqueName: \"kubernetes.io/projected/6d470205-91ef-49c4-a40e-fc5a2d131ddb-kube-api-access-8cvb7\") pod \"console-5b85f648d5-hcrv5\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:02.988939 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:02.988918 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:03.113245 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:03.113224 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b85f648d5-hcrv5"] Jul 28 14:28:03.118628 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:28:03.118592 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d470205_91ef_49c4_a40e_fc5a2d131ddb.slice/crio-b0029b64cef54e5c8296114346bba2437adbb08a023fad0f13f18d9ed7ee55ab WatchSource:0}: Error finding container b0029b64cef54e5c8296114346bba2437adbb08a023fad0f13f18d9ed7ee55ab: Status 404 returned error can't find the container with id b0029b64cef54e5c8296114346bba2437adbb08a023fad0f13f18d9ed7ee55ab Jul 28 14:28:04.075673 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:04.075639 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85f648d5-hcrv5" event={"ID":"6d470205-91ef-49c4-a40e-fc5a2d131ddb","Type":"ContainerStarted","Data":"2436908fb249884aa6fed5a59086f1026188d92cef58b010dacd322c355a9492"} Jul 28 14:28:04.076087 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:04.075685 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85f648d5-hcrv5" event={"ID":"6d470205-91ef-49c4-a40e-fc5a2d131ddb","Type":"ContainerStarted","Data":"b0029b64cef54e5c8296114346bba2437adbb08a023fad0f13f18d9ed7ee55ab"} Jul 28 14:28:04.093152 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:04.093108 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b85f648d5-hcrv5" podStartSLOduration=2.093093811 podStartE2EDuration="2.093093811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:28:04.091875557 +0000 UTC m=+314.892753113" watchObservedRunningTime="2026-07-28 14:28:04.093093811 +0000 UTC m=+314.893971366" Jul 28 14:28:12.989873 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:12.989837 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:12.990276 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:12.989915 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:12.994801 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:12.994778 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:13.115962 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:13.115937 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:28:13.162452 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:13.162420 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8d87fc8d6-fhxtq"] Jul 28 14:28:38.183188 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.183132 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8d87fc8d6-fhxtq" podUID="291758e4-a31f-4d4e-8c24-46306a15afee" containerName="console" containerID="cri-o://76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf" gracePeriod=15 Jul 28 14:28:38.419940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.419915 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:28:38.467569 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467502 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-console-config\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.467569 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467532 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-oauth-config\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.467751 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467578 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-trusted-ca-bundle\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.467751 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467608 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-service-ca\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.467751 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467643 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-serving-cert\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.467751 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467684 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-oauth-serving-cert\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.467751 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467721 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgj9f\" (UniqueName: \"kubernetes.io/projected/291758e4-a31f-4d4e-8c24-46306a15afee-kube-api-access-dgj9f\") pod \"291758e4-a31f-4d4e-8c24-46306a15afee\" (UID: \"291758e4-a31f-4d4e-8c24-46306a15afee\") " Jul 28 14:28:38.468012 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467828 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-console-config" (OuterVolumeSpecName: "console-config") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:28:38.468012 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.467939 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-console-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:38.468115 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.468034 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-service-ca" (OuterVolumeSpecName: "service-ca") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:28:38.468115 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.468069 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:28:38.468115 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.468079 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:28:38.469524 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.469503 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:28:38.469693 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.469674 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291758e4-a31f-4d4e-8c24-46306a15afee-kube-api-access-dgj9f" (OuterVolumeSpecName: "kube-api-access-dgj9f") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "kube-api-access-dgj9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:28:38.469740 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.469731 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "291758e4-a31f-4d4e-8c24-46306a15afee" (UID: "291758e4-a31f-4d4e-8c24-46306a15afee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:28:38.568958 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.568935 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:38.568958 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.568956 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-oauth-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:38.569076 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.568965 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgj9f\" (UniqueName: \"kubernetes.io/projected/291758e4-a31f-4d4e-8c24-46306a15afee-kube-api-access-dgj9f\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:38.569076 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.568975 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/291758e4-a31f-4d4e-8c24-46306a15afee-console-oauth-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:38.569076 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.568984 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-trusted-ca-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:38.569076 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:38.568993 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/291758e4-a31f-4d4e-8c24-46306a15afee-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:28:39.218160 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.218127 2586 generic.go:356] "Generic (PLEG): container finished" podID="291758e4-a31f-4d4e-8c24-46306a15afee" containerID="76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf" exitCode=0 Jul 28 14:28:39.218641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.218167 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d87fc8d6-fhxtq" Jul 28 14:28:39.218641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.218214 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d87fc8d6-fhxtq" event={"ID":"291758e4-a31f-4d4e-8c24-46306a15afee","Type":"ContainerDied","Data":"76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf"} Jul 28 14:28:39.218641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.218269 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d87fc8d6-fhxtq" event={"ID":"291758e4-a31f-4d4e-8c24-46306a15afee","Type":"ContainerDied","Data":"890f53463097020f83a8abc30042f1f0bd7edf1392d8ae97d182925511386c60"} Jul 28 14:28:39.218641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.218312 2586 scope.go:117] "RemoveContainer" containerID="76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf" Jul 28 14:28:39.227049 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.227036 2586 scope.go:117] "RemoveContainer" containerID="76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf" Jul 28 14:28:39.227308 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:28:39.227288 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf\": container with ID starting with 76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf not found: ID does not exist" containerID="76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf" Jul 28 14:28:39.227362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.227316 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf"} err="failed to get container status \"76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf\": rpc error: code = NotFound desc = could not find container \"76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf\": container with ID starting with 76e1c240225ee56f7ff741313bffb1c2e8312903b307c7dba57004578dcf08cf not found: ID does not exist" Jul 28 14:28:39.239028 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.239007 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8d87fc8d6-fhxtq"] Jul 28 14:28:39.245331 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.245314 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8d87fc8d6-fhxtq"] Jul 28 14:28:39.690390 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:39.690353 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291758e4-a31f-4d4e-8c24-46306a15afee" path="/var/lib/kubelet/pods/291758e4-a31f-4d4e-8c24-46306a15afee/volumes" Jul 28 14:28:57.902115 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.902080 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54"] Jul 28 14:28:57.902455 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.902429 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291758e4-a31f-4d4e-8c24-46306a15afee" containerName="console" Jul 28 14:28:57.902455 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.902441 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="291758e4-a31f-4d4e-8c24-46306a15afee" containerName="console" Jul 28 14:28:57.902542 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.902515 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="291758e4-a31f-4d4e-8c24-46306a15afee" containerName="console" Jul 28 14:28:57.906742 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.906726 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:57.909310 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.909290 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 14:28:57.909396 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.909328 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 14:28:57.910362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.910289 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s57xl\"" Jul 28 14:28:57.912005 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:57.911984 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54"] Jul 28 14:28:58.009821 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.009792 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.009963 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.009828 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wk5b\" (UniqueName: \"kubernetes.io/projected/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-kube-api-access-8wk5b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.009963 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.009882 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.110289 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.110258 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.110289 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.110292 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wk5b\" (UniqueName: \"kubernetes.io/projected/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-kube-api-access-8wk5b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.110484 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.110332 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.110664 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.110646 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.110710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.110679 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.118141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.118116 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wk5b\" (UniqueName: \"kubernetes.io/projected/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-kube-api-access-8wk5b\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.217340 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.217267 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:28:58.348396 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.348365 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54"] Jul 28 14:28:58.351278 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:28:58.351252 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13cf7e5f_d947_42ed_9ad5_7d4e8f5ecbed.slice/crio-221aacf4d98081b855a3c3a7e1b0f7127db98f41d02d46206936f53b7c1d9f95 WatchSource:0}: Error finding container 221aacf4d98081b855a3c3a7e1b0f7127db98f41d02d46206936f53b7c1d9f95: Status 404 returned error can't find the container with id 221aacf4d98081b855a3c3a7e1b0f7127db98f41d02d46206936f53b7c1d9f95 Jul 28 14:28:58.353109 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:58.353094 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 14:28:59.298092 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:28:59.298055 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" event={"ID":"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed","Type":"ContainerStarted","Data":"221aacf4d98081b855a3c3a7e1b0f7127db98f41d02d46206936f53b7c1d9f95"} Jul 28 14:29:04.327025 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:04.326991 2586 generic.go:356] "Generic (PLEG): container finished" podID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerID="9627676dc2eec7bce686c087862d23fb94956bed768c86acf743792e4a6e45a2" exitCode=0 Jul 28 14:29:04.327394 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:04.327077 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" event={"ID":"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed","Type":"ContainerDied","Data":"9627676dc2eec7bce686c087862d23fb94956bed768c86acf743792e4a6e45a2"} Jul 28 14:29:07.343102 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:07.343071 2586 generic.go:356] "Generic (PLEG): container finished" podID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerID="67d5c7236463c221d3674d7dfcabf92e3bbb3736c0beca3998d480cc8600bc7c" exitCode=0 Jul 28 14:29:07.343497 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:07.343158 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" event={"ID":"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed","Type":"ContainerDied","Data":"67d5c7236463c221d3674d7dfcabf92e3bbb3736c0beca3998d480cc8600bc7c"} Jul 28 14:29:15.378138 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:15.378098 2586 generic.go:356] "Generic (PLEG): container finished" podID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerID="7f30d97d2866a8fb3b986d92d2405ad958744e3dfe3da730b38b145d0e72e33d" exitCode=0 Jul 28 14:29:15.378534 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:15.378186 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" event={"ID":"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed","Type":"ContainerDied","Data":"7f30d97d2866a8fb3b986d92d2405ad958744e3dfe3da730b38b145d0e72e33d"} Jul 28 14:29:16.503524 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.503500 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:29:16.572779 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.572748 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-bundle\") pod \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " Jul 28 14:29:16.572931 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.572822 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-util\") pod \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " Jul 28 14:29:16.572931 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.572876 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wk5b\" (UniqueName: \"kubernetes.io/projected/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-kube-api-access-8wk5b\") pod \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\" (UID: \"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed\") " Jul 28 14:29:16.573494 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.573473 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-bundle" (OuterVolumeSpecName: "bundle") pod "13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" (UID: "13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:29:16.575040 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.575006 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-kube-api-access-8wk5b" (OuterVolumeSpecName: "kube-api-access-8wk5b") pod "13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" (UID: "13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed"). InnerVolumeSpecName "kube-api-access-8wk5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:29:16.576937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.576916 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-util" (OuterVolumeSpecName: "util") pod "13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" (UID: "13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:29:16.674129 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.674038 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wk5b\" (UniqueName: \"kubernetes.io/projected/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-kube-api-access-8wk5b\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:16.674129 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.674079 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:16.674129 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:16.674090 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:17.387514 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:17.387471 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" event={"ID":"13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed","Type":"ContainerDied","Data":"221aacf4d98081b855a3c3a7e1b0f7127db98f41d02d46206936f53b7c1d9f95"} Jul 28 14:29:17.387713 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:17.387528 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="221aacf4d98081b855a3c3a7e1b0f7127db98f41d02d46206936f53b7c1d9f95" Jul 28 14:29:17.387713 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:17.387496 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540frg54" Jul 28 14:29:20.862630 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.862602 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr"] Jul 28 14:29:20.862993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.862954 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="util" Jul 28 14:29:20.862993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.862966 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="util" Jul 28 14:29:20.862993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.862978 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="pull" Jul 28 14:29:20.862993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.862986 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="pull" Jul 28 14:29:20.863123 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.863000 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="extract" Jul 28 14:29:20.863123 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.863006 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="extract" Jul 28 14:29:20.863123 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.863061 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="13cf7e5f-d947-42ed-9ad5-7d4e8f5ecbed" containerName="extract" Jul 28 14:29:20.905086 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.905055 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr"] Jul 28 14:29:20.905263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.905141 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:20.908106 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.908080 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 28 14:29:20.908237 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.908082 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:29:20.908237 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:20.908122 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-j2ph6\"" Jul 28 14:29:21.011240 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.011201 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2cd8c0e-a309-4b48-b81e-32a39d62165b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gttjr\" (UID: \"d2cd8c0e-a309-4b48-b81e-32a39d62165b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.011407 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.011247 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26nhn\" (UniqueName: \"kubernetes.io/projected/d2cd8c0e-a309-4b48-b81e-32a39d62165b-kube-api-access-26nhn\") pod \"cert-manager-operator-controller-manager-55c444bf86-gttjr\" (UID: \"d2cd8c0e-a309-4b48-b81e-32a39d62165b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.112135 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.112102 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2cd8c0e-a309-4b48-b81e-32a39d62165b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gttjr\" (UID: \"d2cd8c0e-a309-4b48-b81e-32a39d62165b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.112135 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.112138 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26nhn\" (UniqueName: \"kubernetes.io/projected/d2cd8c0e-a309-4b48-b81e-32a39d62165b-kube-api-access-26nhn\") pod \"cert-manager-operator-controller-manager-55c444bf86-gttjr\" (UID: \"d2cd8c0e-a309-4b48-b81e-32a39d62165b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.112488 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.112471 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2cd8c0e-a309-4b48-b81e-32a39d62165b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gttjr\" (UID: \"d2cd8c0e-a309-4b48-b81e-32a39d62165b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.121744 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.121688 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26nhn\" (UniqueName: \"kubernetes.io/projected/d2cd8c0e-a309-4b48-b81e-32a39d62165b-kube-api-access-26nhn\") pod \"cert-manager-operator-controller-manager-55c444bf86-gttjr\" (UID: \"d2cd8c0e-a309-4b48-b81e-32a39d62165b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.214236 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.214203 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" Jul 28 14:29:21.360875 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.360852 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr"] Jul 28 14:29:21.363657 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:21.363632 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cd8c0e_a309_4b48_b81e_32a39d62165b.slice/crio-53232ea3428a1ba8b828ae7d4cb6ba18d3d7e9b0eef7d9de8b2feec3f253d262 WatchSource:0}: Error finding container 53232ea3428a1ba8b828ae7d4cb6ba18d3d7e9b0eef7d9de8b2feec3f253d262: Status 404 returned error can't find the container with id 53232ea3428a1ba8b828ae7d4cb6ba18d3d7e9b0eef7d9de8b2feec3f253d262 Jul 28 14:29:21.404799 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:21.404733 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" event={"ID":"d2cd8c0e-a309-4b48-b81e-32a39d62165b","Type":"ContainerStarted","Data":"53232ea3428a1ba8b828ae7d4cb6ba18d3d7e9b0eef7d9de8b2feec3f253d262"} Jul 28 14:29:24.420060 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:24.420024 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" event={"ID":"d2cd8c0e-a309-4b48-b81e-32a39d62165b","Type":"ContainerStarted","Data":"8bdf78b6ac4125f656491f9028f1a6139dd0e55799b164c23297ec2e31c42cd5"} Jul 28 14:29:24.439940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:24.439896 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gttjr" podStartSLOduration=1.970518266 podStartE2EDuration="4.439876894s" totalImagesPullingTime="2.469358628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:29:24.43931604 +0000 UTC m=+395.240193610" watchObservedRunningTime="2026-07-28 14:29:24.439876894 +0000 UTC m=+395.240754451" Jul 28 14:29:25.086650 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.086620 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc"] Jul 28 14:29:25.107322 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.107300 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc"] Jul 28 14:29:25.107471 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.107389 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.109919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.109894 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 14:29:25.110052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.109933 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 14:29:25.110699 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.110681 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s57xl\"" Jul 28 14:29:25.121857 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.121837 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jvd2l"] Jul 28 14:29:25.134020 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.134001 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.136446 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.136428 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bf9mt\"" Jul 28 14:29:25.136750 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.136735 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 28 14:29:25.136833 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.136796 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jvd2l"] Jul 28 14:29:25.137588 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.137575 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 28 14:29:25.248719 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.248681 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7z89\" (UniqueName: \"kubernetes.io/projected/3464be06-4345-4907-83bd-f52072ce4ff7-kube-api-access-q7z89\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.248897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.248729 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2xmv\" (UniqueName: \"kubernetes.io/projected/9df15c30-81c7-4ccf-87fb-41377663b211-kube-api-access-w2xmv\") pod \"cert-manager-webhook-6486ffd997-jvd2l\" (UID: \"9df15c30-81c7-4ccf-87fb-41377663b211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.248897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.248764 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.248897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.248780 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9df15c30-81c7-4ccf-87fb-41377663b211-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jvd2l\" (UID: \"9df15c30-81c7-4ccf-87fb-41377663b211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.248897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.248804 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.350019 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.349938 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.350019 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.349994 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7z89\" (UniqueName: \"kubernetes.io/projected/3464be06-4345-4907-83bd-f52072ce4ff7-kube-api-access-q7z89\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.350241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.350025 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2xmv\" (UniqueName: \"kubernetes.io/projected/9df15c30-81c7-4ccf-87fb-41377663b211-kube-api-access-w2xmv\") pod \"cert-manager-webhook-6486ffd997-jvd2l\" (UID: \"9df15c30-81c7-4ccf-87fb-41377663b211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.350241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.350057 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.350241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.350076 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9df15c30-81c7-4ccf-87fb-41377663b211-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jvd2l\" (UID: \"9df15c30-81c7-4ccf-87fb-41377663b211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.350434 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.350411 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.350488 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.350457 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.358804 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.358777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2xmv\" (UniqueName: \"kubernetes.io/projected/9df15c30-81c7-4ccf-87fb-41377663b211-kube-api-access-w2xmv\") pod \"cert-manager-webhook-6486ffd997-jvd2l\" (UID: \"9df15c30-81c7-4ccf-87fb-41377663b211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.359422 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.359395 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9df15c30-81c7-4ccf-87fb-41377663b211-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jvd2l\" (UID: \"9df15c30-81c7-4ccf-87fb-41377663b211\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.363297 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.359740 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7z89\" (UniqueName: \"kubernetes.io/projected/3464be06-4345-4907-83bd-f52072ce4ff7-kube-api-access-q7z89\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.417503 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.417474 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:25.459004 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.458977 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:25.567909 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.567882 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc"] Jul 28 14:29:25.569887 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:25.569854 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3464be06_4345_4907_83bd_f52072ce4ff7.slice/crio-970564ea292b40492625a9d2948e385578323a99a0be11eef960193ec880d81b WatchSource:0}: Error finding container 970564ea292b40492625a9d2948e385578323a99a0be11eef960193ec880d81b: Status 404 returned error can't find the container with id 970564ea292b40492625a9d2948e385578323a99a0be11eef960193ec880d81b Jul 28 14:29:25.610791 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:25.610722 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jvd2l"] Jul 28 14:29:25.622644 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:25.622618 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9df15c30_81c7_4ccf_87fb_41377663b211.slice/crio-4e920cbf3e64d71290e68957be7de79b1310019c8abee33f6af1303fabf1179a WatchSource:0}: Error finding container 4e920cbf3e64d71290e68957be7de79b1310019c8abee33f6af1303fabf1179a: Status 404 returned error can't find the container with id 4e920cbf3e64d71290e68957be7de79b1310019c8abee33f6af1303fabf1179a Jul 28 14:29:26.430417 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:26.430376 2586 generic.go:356] "Generic (PLEG): container finished" podID="3464be06-4345-4907-83bd-f52072ce4ff7" containerID="304790c130bc5f8ce9a899844280f8b9cb224c876aa51bdde9fb2224feb86984" exitCode=0 Jul 28 14:29:26.430621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:26.430465 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" event={"ID":"3464be06-4345-4907-83bd-f52072ce4ff7","Type":"ContainerDied","Data":"304790c130bc5f8ce9a899844280f8b9cb224c876aa51bdde9fb2224feb86984"} Jul 28 14:29:26.430621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:26.430520 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" event={"ID":"3464be06-4345-4907-83bd-f52072ce4ff7","Type":"ContainerStarted","Data":"970564ea292b40492625a9d2948e385578323a99a0be11eef960193ec880d81b"} Jul 28 14:29:26.431691 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:26.431661 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" event={"ID":"9df15c30-81c7-4ccf-87fb-41377663b211","Type":"ContainerStarted","Data":"4e920cbf3e64d71290e68957be7de79b1310019c8abee33f6af1303fabf1179a"} Jul 28 14:29:30.455086 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:30.455047 2586 generic.go:356] "Generic (PLEG): container finished" podID="3464be06-4345-4907-83bd-f52072ce4ff7" containerID="25bd7236f1008668eb5fbc4b5754aea0ee387d6616e35d4275b2472776f3ca99" exitCode=0 Jul 28 14:29:30.455520 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:30.455126 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" event={"ID":"3464be06-4345-4907-83bd-f52072ce4ff7","Type":"ContainerDied","Data":"25bd7236f1008668eb5fbc4b5754aea0ee387d6616e35d4275b2472776f3ca99"} Jul 28 14:29:30.456715 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:30.456691 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" event={"ID":"9df15c30-81c7-4ccf-87fb-41377663b211","Type":"ContainerStarted","Data":"16f5aaa0504ab9f7f0a25be89d56b0695ab5736e767a9c8aacbd77958ee5577b"} Jul 28 14:29:30.456793 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:30.456775 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:30.500668 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:30.500627 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" podStartSLOduration=1.704360871 podStartE2EDuration="5.500611596s" totalImagesPullingTime="3.796250725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:29:30.500172772 +0000 UTC m=+401.301050329" watchObservedRunningTime="2026-07-28 14:29:30.500611596 +0000 UTC m=+401.301489153" Jul 28 14:29:31.463494 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:31.463452 2586 generic.go:356] "Generic (PLEG): container finished" podID="3464be06-4345-4907-83bd-f52072ce4ff7" containerID="8fadd254e38ecf5ebf09b05936341310ac6ca2e5b6d229155e6d4376bcc659bd" exitCode=0 Jul 28 14:29:31.463917 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:31.463584 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" event={"ID":"3464be06-4345-4907-83bd-f52072ce4ff7","Type":"ContainerDied","Data":"8fadd254e38ecf5ebf09b05936341310ac6ca2e5b6d229155e6d4376bcc659bd"} Jul 28 14:29:32.591829 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.591804 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:32.611945 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.611914 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-util\") pod \"3464be06-4345-4907-83bd-f52072ce4ff7\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " Jul 28 14:29:32.612075 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.612001 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7z89\" (UniqueName: \"kubernetes.io/projected/3464be06-4345-4907-83bd-f52072ce4ff7-kube-api-access-q7z89\") pod \"3464be06-4345-4907-83bd-f52072ce4ff7\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " Jul 28 14:29:32.612075 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.612035 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-bundle\") pod \"3464be06-4345-4907-83bd-f52072ce4ff7\" (UID: \"3464be06-4345-4907-83bd-f52072ce4ff7\") " Jul 28 14:29:32.612590 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.612566 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-bundle" (OuterVolumeSpecName: "bundle") pod "3464be06-4345-4907-83bd-f52072ce4ff7" (UID: "3464be06-4345-4907-83bd-f52072ce4ff7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:29:32.614088 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.614065 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3464be06-4345-4907-83bd-f52072ce4ff7-kube-api-access-q7z89" (OuterVolumeSpecName: "kube-api-access-q7z89") pod "3464be06-4345-4907-83bd-f52072ce4ff7" (UID: "3464be06-4345-4907-83bd-f52072ce4ff7"). InnerVolumeSpecName "kube-api-access-q7z89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:29:32.616997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.616964 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-util" (OuterVolumeSpecName: "util") pod "3464be06-4345-4907-83bd-f52072ce4ff7" (UID: "3464be06-4345-4907-83bd-f52072ce4ff7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:29:32.712746 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.712713 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:32.712746 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.712740 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7z89\" (UniqueName: \"kubernetes.io/projected/3464be06-4345-4907-83bd-f52072ce4ff7-kube-api-access-q7z89\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:32.712746 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:32.712750 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3464be06-4345-4907-83bd-f52072ce4ff7-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:33.474068 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:33.474037 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" Jul 28 14:29:33.474068 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:33.474045 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq5jpc" event={"ID":"3464be06-4345-4907-83bd-f52072ce4ff7","Type":"ContainerDied","Data":"970564ea292b40492625a9d2948e385578323a99a0be11eef960193ec880d81b"} Jul 28 14:29:33.474263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:33.474084 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="970564ea292b40492625a9d2948e385578323a99a0be11eef960193ec880d81b" Jul 28 14:29:35.104453 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104421 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ktnnt"] Jul 28 14:29:35.104842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104788 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="util" Jul 28 14:29:35.104842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104799 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="util" Jul 28 14:29:35.104842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104812 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="extract" Jul 28 14:29:35.104842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104818 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="extract" Jul 28 14:29:35.104842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104839 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="pull" Jul 28 14:29:35.104842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104845 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="pull" Jul 28 14:29:35.105023 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.104890 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3464be06-4345-4907-83bd-f52072ce4ff7" containerName="extract" Jul 28 14:29:35.106845 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.106830 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.109092 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.109071 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-b2sjw\"" Jul 28 14:29:35.117163 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.117142 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ktnnt"] Jul 28 14:29:35.130127 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.130102 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/847eeefc-7ac5-440c-b6f6-3106e9081603-bound-sa-token\") pod \"cert-manager-55b4d954d5-ktnnt\" (UID: \"847eeefc-7ac5-440c-b6f6-3106e9081603\") " pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.130273 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.130252 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lls49\" (UniqueName: \"kubernetes.io/projected/847eeefc-7ac5-440c-b6f6-3106e9081603-kube-api-access-lls49\") pod \"cert-manager-55b4d954d5-ktnnt\" (UID: \"847eeefc-7ac5-440c-b6f6-3106e9081603\") " pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.231530 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.231491 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/847eeefc-7ac5-440c-b6f6-3106e9081603-bound-sa-token\") pod \"cert-manager-55b4d954d5-ktnnt\" (UID: \"847eeefc-7ac5-440c-b6f6-3106e9081603\") " pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.231722 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.231607 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lls49\" (UniqueName: \"kubernetes.io/projected/847eeefc-7ac5-440c-b6f6-3106e9081603-kube-api-access-lls49\") pod \"cert-manager-55b4d954d5-ktnnt\" (UID: \"847eeefc-7ac5-440c-b6f6-3106e9081603\") " pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.239940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.239907 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/847eeefc-7ac5-440c-b6f6-3106e9081603-bound-sa-token\") pod \"cert-manager-55b4d954d5-ktnnt\" (UID: \"847eeefc-7ac5-440c-b6f6-3106e9081603\") " pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.240060 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.239985 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lls49\" (UniqueName: \"kubernetes.io/projected/847eeefc-7ac5-440c-b6f6-3106e9081603-kube-api-access-lls49\") pod \"cert-manager-55b4d954d5-ktnnt\" (UID: \"847eeefc-7ac5-440c-b6f6-3106e9081603\") " pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.416241 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.416150 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ktnnt" Jul 28 14:29:35.541897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:35.541874 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ktnnt"] Jul 28 14:29:35.544361 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:35.544328 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod847eeefc_7ac5_440c_b6f6_3106e9081603.slice/crio-6ed6d10649f5b3893414cf1ea1dba3f7dbe806ebf3eb431e5607043637a95f2b WatchSource:0}: Error finding container 6ed6d10649f5b3893414cf1ea1dba3f7dbe806ebf3eb431e5607043637a95f2b: Status 404 returned error can't find the container with id 6ed6d10649f5b3893414cf1ea1dba3f7dbe806ebf3eb431e5607043637a95f2b Jul 28 14:29:36.465785 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:36.465750 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jvd2l" Jul 28 14:29:36.489985 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:36.489951 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ktnnt" event={"ID":"847eeefc-7ac5-440c-b6f6-3106e9081603","Type":"ContainerStarted","Data":"cb97e47e3a205d98a82a4f8c0bc1d653ebb127e3bf98d52663e531665313cd28"} Jul 28 14:29:36.490141 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:36.489995 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ktnnt" event={"ID":"847eeefc-7ac5-440c-b6f6-3106e9081603","Type":"ContainerStarted","Data":"6ed6d10649f5b3893414cf1ea1dba3f7dbe806ebf3eb431e5607043637a95f2b"} Jul 28 14:29:36.506397 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:36.506344 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ktnnt" podStartSLOduration=1.506326601 podStartE2EDuration="1.506326601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:29:36.504936293 +0000 UTC m=+407.305813852" watchObservedRunningTime="2026-07-28 14:29:36.506326601 +0000 UTC m=+407.307204157" Jul 28 14:29:42.966492 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.966463 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6"] Jul 28 14:29:42.969143 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.969127 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:42.971950 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.971926 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 28 14:29:42.971950 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.971936 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 28 14:29:42.972722 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.972703 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gw68l\"" Jul 28 14:29:42.979340 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.979321 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6"] Jul 28 14:29:42.984367 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.984337 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d791c6a0-0447-47e5-a606-33c0f21d5c80-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gvhf6\" (UID: \"d791c6a0-0447-47e5-a606-33c0f21d5c80\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:42.984469 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:42.984380 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbbxx\" (UniqueName: \"kubernetes.io/projected/d791c6a0-0447-47e5-a606-33c0f21d5c80-kube-api-access-xbbxx\") pod \"openshift-lws-operator-fd8ccff4c-gvhf6\" (UID: \"d791c6a0-0447-47e5-a606-33c0f21d5c80\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:43.084910 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.084888 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbbxx\" (UniqueName: \"kubernetes.io/projected/d791c6a0-0447-47e5-a606-33c0f21d5c80-kube-api-access-xbbxx\") pod \"openshift-lws-operator-fd8ccff4c-gvhf6\" (UID: \"d791c6a0-0447-47e5-a606-33c0f21d5c80\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:43.085002 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.084952 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d791c6a0-0447-47e5-a606-33c0f21d5c80-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gvhf6\" (UID: \"d791c6a0-0447-47e5-a606-33c0f21d5c80\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:43.085237 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.085222 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d791c6a0-0447-47e5-a606-33c0f21d5c80-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gvhf6\" (UID: \"d791c6a0-0447-47e5-a606-33c0f21d5c80\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:43.092583 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.092541 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbbxx\" (UniqueName: \"kubernetes.io/projected/d791c6a0-0447-47e5-a606-33c0f21d5c80-kube-api-access-xbbxx\") pod \"openshift-lws-operator-fd8ccff4c-gvhf6\" (UID: \"d791c6a0-0447-47e5-a606-33c0f21d5c80\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:43.278842 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.278787 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" Jul 28 14:29:43.403116 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.403095 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6"] Jul 28 14:29:43.405271 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:43.405246 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd791c6a0_0447_47e5_a606_33c0f21d5c80.slice/crio-bc7ac30e7679aaff278f4ef4abf9128817f582603f769330f251474661a4f6df WatchSource:0}: Error finding container bc7ac30e7679aaff278f4ef4abf9128817f582603f769330f251474661a4f6df: Status 404 returned error can't find the container with id bc7ac30e7679aaff278f4ef4abf9128817f582603f769330f251474661a4f6df Jul 28 14:29:43.521092 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:43.521063 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" event={"ID":"d791c6a0-0447-47e5-a606-33c0f21d5c80","Type":"ContainerStarted","Data":"bc7ac30e7679aaff278f4ef4abf9128817f582603f769330f251474661a4f6df"} Jul 28 14:29:45.532752 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:45.532669 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" event={"ID":"d791c6a0-0447-47e5-a606-33c0f21d5c80","Type":"ContainerStarted","Data":"5b327b8eec49e7cfbe782fdc3ae57aea66d6e0202bb61b14fb522c409705025d"} Jul 28 14:29:45.548595 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:45.548531 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvhf6" podStartSLOduration=1.824254281 podStartE2EDuration="3.548511644s" totalImagesPullingTime="1.724257363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:29:45.547676651 +0000 UTC m=+416.348554207" watchObservedRunningTime="2026-07-28 14:29:45.548511644 +0000 UTC m=+416.349389192" Jul 28 14:29:50.584822 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.584784 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2"] Jul 28 14:29:50.587403 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.587384 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.589783 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.589766 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s57xl\"" Jul 28 14:29:50.589989 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.589976 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 14:29:50.590748 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.590736 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 14:29:50.596038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.596018 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2"] Jul 28 14:29:50.635290 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.635265 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n75x\" (UniqueName: \"kubernetes.io/projected/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-kube-api-access-4n75x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.635389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.635318 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.635389 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.635361 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.735711 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.735684 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n75x\" (UniqueName: \"kubernetes.io/projected/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-kube-api-access-4n75x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.735803 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.735740 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.735803 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.735776 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.736072 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.736058 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.736110 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.736092 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.744037 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.744014 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n75x\" (UniqueName: \"kubernetes.io/projected/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-kube-api-access-4n75x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:50.896038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:50.895987 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:51.023119 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:51.023095 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2"] Jul 28 14:29:51.025708 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:51.025683 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a86a9da_ac37_46d5_b979_ac18a62a5b4f.slice/crio-f39c94a132676d734ceb23b57cf9cf18e66ec90ab3822b52a79f4f17cf585997 WatchSource:0}: Error finding container f39c94a132676d734ceb23b57cf9cf18e66ec90ab3822b52a79f4f17cf585997: Status 404 returned error can't find the container with id f39c94a132676d734ceb23b57cf9cf18e66ec90ab3822b52a79f4f17cf585997 Jul 28 14:29:51.561202 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:51.561173 2586 generic.go:356] "Generic (PLEG): container finished" podID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerID="90b8948e56dade0dac12c279a358758f0d531c2cfeed7a5469121aa1d352f826" exitCode=0 Jul 28 14:29:51.561356 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:51.561259 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" event={"ID":"6a86a9da-ac37-46d5-b979-ac18a62a5b4f","Type":"ContainerDied","Data":"90b8948e56dade0dac12c279a358758f0d531c2cfeed7a5469121aa1d352f826"} Jul 28 14:29:51.561356 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:51.561307 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" event={"ID":"6a86a9da-ac37-46d5-b979-ac18a62a5b4f","Type":"ContainerStarted","Data":"f39c94a132676d734ceb23b57cf9cf18e66ec90ab3822b52a79f4f17cf585997"} Jul 28 14:29:52.567791 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:52.567764 2586 generic.go:356] "Generic (PLEG): container finished" podID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerID="560c6dbc2356f976506496650184cd53bc8e5f6d108ef655ef35fd6517c47750" exitCode=0 Jul 28 14:29:52.568115 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:52.567826 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" event={"ID":"6a86a9da-ac37-46d5-b979-ac18a62a5b4f","Type":"ContainerDied","Data":"560c6dbc2356f976506496650184cd53bc8e5f6d108ef655ef35fd6517c47750"} Jul 28 14:29:53.575365 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:53.575333 2586 generic.go:356] "Generic (PLEG): container finished" podID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerID="23259176341e19ffd2b36ce12eba4dbd8be573475f726d1cecdf59f51ea2614c" exitCode=0 Jul 28 14:29:53.575365 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:53.575372 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" event={"ID":"6a86a9da-ac37-46d5-b979-ac18a62a5b4f","Type":"ContainerDied","Data":"23259176341e19ffd2b36ce12eba4dbd8be573475f726d1cecdf59f51ea2614c"} Jul 28 14:29:54.713095 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.713074 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:54.762296 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.762273 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-bundle\") pod \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " Jul 28 14:29:54.762417 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.762334 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-util\") pod \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " Jul 28 14:29:54.762417 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.762365 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n75x\" (UniqueName: \"kubernetes.io/projected/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-kube-api-access-4n75x\") pod \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\" (UID: \"6a86a9da-ac37-46d5-b979-ac18a62a5b4f\") " Jul 28 14:29:54.763036 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.763014 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-bundle" (OuterVolumeSpecName: "bundle") pod "6a86a9da-ac37-46d5-b979-ac18a62a5b4f" (UID: "6a86a9da-ac37-46d5-b979-ac18a62a5b4f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:29:54.764438 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.764414 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-kube-api-access-4n75x" (OuterVolumeSpecName: "kube-api-access-4n75x") pod "6a86a9da-ac37-46d5-b979-ac18a62a5b4f" (UID: "6a86a9da-ac37-46d5-b979-ac18a62a5b4f"). InnerVolumeSpecName "kube-api-access-4n75x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:29:54.767750 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.767719 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-util" (OuterVolumeSpecName: "util") pod "6a86a9da-ac37-46d5-b979-ac18a62a5b4f" (UID: "6a86a9da-ac37-46d5-b979-ac18a62a5b4f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:29:54.863124 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.863055 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:54.863124 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.863091 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n75x\" (UniqueName: \"kubernetes.io/projected/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-kube-api-access-4n75x\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:54.863124 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:54.863105 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a86a9da-ac37-46d5-b979-ac18a62a5b4f-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:29:55.586395 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:55.586360 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" Jul 28 14:29:55.586395 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:55.586390 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afh4mg2" event={"ID":"6a86a9da-ac37-46d5-b979-ac18a62a5b4f","Type":"ContainerDied","Data":"f39c94a132676d734ceb23b57cf9cf18e66ec90ab3822b52a79f4f17cf585997"} Jul 28 14:29:55.586626 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:55.586435 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f39c94a132676d734ceb23b57cf9cf18e66ec90ab3822b52a79f4f17cf585997" Jul 28 14:29:56.621231 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621202 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds"] Jul 28 14:29:56.621621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621533 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="pull" Jul 28 14:29:56.621621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621560 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="pull" Jul 28 14:29:56.621621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621579 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="extract" Jul 28 14:29:56.621621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621588 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="extract" Jul 28 14:29:56.621621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621609 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="util" Jul 28 14:29:56.621621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621617 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="util" Jul 28 14:29:56.621807 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.621700 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a86a9da-ac37-46d5-b979-ac18a62a5b4f" containerName="extract" Jul 28 14:29:56.625493 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.625477 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.631398 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.631375 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 28 14:29:56.631513 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.631398 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jfjsq\"" Jul 28 14:29:56.631513 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.631421 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 28 14:29:56.631513 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.631473 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 28 14:29:56.658725 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.658704 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds"] Jul 28 14:29:56.674481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.674457 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t9tj\" (UniqueName: \"kubernetes.io/projected/5df1bc09-634a-4bac-8917-6e2a20aefe19-kube-api-access-5t9tj\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.674598 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.674503 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5df1bc09-634a-4bac-8917-6e2a20aefe19-manager-config\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.674598 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.674532 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df1bc09-634a-4bac-8917-6e2a20aefe19-cert\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.674731 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.674641 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5df1bc09-634a-4bac-8917-6e2a20aefe19-metrics-cert\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.775189 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.775170 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t9tj\" (UniqueName: \"kubernetes.io/projected/5df1bc09-634a-4bac-8917-6e2a20aefe19-kube-api-access-5t9tj\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.775301 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.775208 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5df1bc09-634a-4bac-8917-6e2a20aefe19-manager-config\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.775301 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.775227 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df1bc09-634a-4bac-8917-6e2a20aefe19-cert\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.775415 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.775378 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5df1bc09-634a-4bac-8917-6e2a20aefe19-metrics-cert\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.775864 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.775838 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5df1bc09-634a-4bac-8917-6e2a20aefe19-manager-config\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.777486 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.777467 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5df1bc09-634a-4bac-8917-6e2a20aefe19-cert\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.777599 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.777583 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5df1bc09-634a-4bac-8917-6e2a20aefe19-metrics-cert\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.783260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.783241 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t9tj\" (UniqueName: \"kubernetes.io/projected/5df1bc09-634a-4bac-8917-6e2a20aefe19-kube-api-access-5t9tj\") pod \"lws-controller-manager-7cf745d86f-ckhds\" (UID: \"5df1bc09-634a-4bac-8917-6e2a20aefe19\") " pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:56.934753 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:56.934696 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:29:57.078270 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:57.078239 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds"] Jul 28 14:29:57.081405 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:29:57.081381 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df1bc09_634a_4bac_8917_6e2a20aefe19.slice/crio-9d0175577887b9ef803651402b2e1c231ec60701949be69d1e0987c45750ebc7 WatchSource:0}: Error finding container 9d0175577887b9ef803651402b2e1c231ec60701949be69d1e0987c45750ebc7: Status 404 returned error can't find the container with id 9d0175577887b9ef803651402b2e1c231ec60701949be69d1e0987c45750ebc7 Jul 28 14:29:57.596986 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:29:57.596952 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" event={"ID":"5df1bc09-634a-4bac-8917-6e2a20aefe19","Type":"ContainerStarted","Data":"9d0175577887b9ef803651402b2e1c231ec60701949be69d1e0987c45750ebc7"} Jul 28 14:30:03.691827 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.691797 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc"] Jul 28 14:30:03.698630 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.698611 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.701144 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.701118 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 28 14:30:03.701621 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.701596 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vfcv9\"" Jul 28 14:30:03.702210 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.701734 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 28 14:30:03.702210 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.701807 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 28 14:30:03.702210 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.702129 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 28 14:30:03.719686 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.719664 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc"] Jul 28 14:30:03.721304 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.721284 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/818d3443-47a8-4e07-a841-77ce24331a17-manifests\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.721413 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.721361 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/818d3443-47a8-4e07-a841-77ce24331a17-tmp\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.721498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.721476 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/818d3443-47a8-4e07-a841-77ce24331a17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.721578 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.721532 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfkl8\" (UniqueName: \"kubernetes.io/projected/818d3443-47a8-4e07-a841-77ce24331a17-kube-api-access-qfkl8\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.721675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.721658 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/818d3443-47a8-4e07-a841-77ce24331a17-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.821997 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.821973 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/818d3443-47a8-4e07-a841-77ce24331a17-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.822090 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.822010 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/818d3443-47a8-4e07-a841-77ce24331a17-manifests\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.822144 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.822127 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/818d3443-47a8-4e07-a841-77ce24331a17-tmp\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.822187 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.822175 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/818d3443-47a8-4e07-a841-77ce24331a17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.822223 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.822205 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfkl8\" (UniqueName: \"kubernetes.io/projected/818d3443-47a8-4e07-a841-77ce24331a17-kube-api-access-qfkl8\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.822345 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.822326 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/818d3443-47a8-4e07-a841-77ce24331a17-manifests\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.824339 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.824316 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/818d3443-47a8-4e07-a841-77ce24331a17-tmp\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.824428 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.824405 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/818d3443-47a8-4e07-a841-77ce24331a17-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.824514 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.824496 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/818d3443-47a8-4e07-a841-77ce24331a17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:03.831801 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:03.831770 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfkl8\" (UniqueName: \"kubernetes.io/projected/818d3443-47a8-4e07-a841-77ce24331a17-kube-api-access-qfkl8\") pod \"opendatahub-operator-controller-manager-6f457fb848-bv5sc\" (UID: \"818d3443-47a8-4e07-a841-77ce24331a17\") " pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:04.012246 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:04.012172 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:04.149822 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:04.149795 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc"] Jul 28 14:30:04.152292 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:30:04.152268 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818d3443_47a8_4e07_a841_77ce24331a17.slice/crio-4ea22993bd7bd7d50981cfc011d0d671ff5e11e19a97496bd87519ef1d32254f WatchSource:0}: Error finding container 4ea22993bd7bd7d50981cfc011d0d671ff5e11e19a97496bd87519ef1d32254f: Status 404 returned error can't find the container with id 4ea22993bd7bd7d50981cfc011d0d671ff5e11e19a97496bd87519ef1d32254f Jul 28 14:30:04.630660 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:04.630627 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" event={"ID":"818d3443-47a8-4e07-a841-77ce24331a17","Type":"ContainerStarted","Data":"4ea22993bd7bd7d50981cfc011d0d671ff5e11e19a97496bd87519ef1d32254f"} Jul 28 14:30:08.655640 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:08.655611 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" event={"ID":"818d3443-47a8-4e07-a841-77ce24331a17","Type":"ContainerStarted","Data":"5a52033de96551ab5b96055d7301be684ec079d1752f7c29074edb9292935738"} Jul 28 14:30:09.663938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:09.663905 2586 generic.go:356] "Generic (PLEG): container finished" podID="818d3443-47a8-4e07-a841-77ce24331a17" containerID="5a52033de96551ab5b96055d7301be684ec079d1752f7c29074edb9292935738" exitCode=0 Jul 28 14:30:09.664329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:09.663994 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" event={"ID":"818d3443-47a8-4e07-a841-77ce24331a17","Type":"ContainerDied","Data":"5a52033de96551ab5b96055d7301be684ec079d1752f7c29074edb9292935738"} Jul 28 14:30:09.665544 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:09.665524 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" event={"ID":"5df1bc09-634a-4bac-8917-6e2a20aefe19","Type":"ContainerStarted","Data":"c061df7c546844840a1174ef1e56715fdc80776b301a2c8a19536a1a7e7ae9a5"} Jul 28 14:30:09.665710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:09.665597 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:30:09.707720 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:09.707666 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" podStartSLOduration=1.773871322 podStartE2EDuration="13.707641067s" totalImagesPullingTime="11.933769745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:09.703803533 +0000 UTC m=+440.504681091" watchObservedRunningTime="2026-07-28 14:30:09.707641067 +0000 UTC m=+440.508518625" Jul 28 14:30:11.677151 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:11.677065 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" event={"ID":"818d3443-47a8-4e07-a841-77ce24331a17","Type":"ContainerStarted","Data":"2708b23903777aee8852dd11370323a0df9ba5950650c3f51bd9791e81f949f3"} Jul 28 14:30:11.677518 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:11.677184 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:11.699122 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:11.697853 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" podStartSLOduration=2.732826705 podStartE2EDuration="8.697831875s" totalImagesPullingTime="4.789446097s" totalInitContainerRuntime="1.175559073s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:30:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:11.696246402 +0000 UTC m=+442.497123965" watchObservedRunningTime="2026-07-28 14:30:11.697831875 +0000 UTC m=+442.498709433" Jul 28 14:30:20.672892 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:20.672859 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7cf745d86f-ckhds" Jul 28 14:30:22.684329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:22.684302 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f457fb848-bv5sc" Jul 28 14:30:32.913249 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.913217 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6"] Jul 28 14:30:32.922542 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.922507 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:32.926401 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.926370 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 28 14:30:32.926666 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.926651 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kvf4n\"" Jul 28 14:30:32.927021 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.927007 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 14:30:32.930983 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.930944 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 14:30:32.940065 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:32.940046 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6"] Jul 28 14:30:33.082883 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.082841 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.082893 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.082920 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bh8h\" (UniqueName: \"kubernetes.io/projected/16174b3f-7b3a-4841-9f09-34edcb19934d-kube-api-access-7bh8h\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.082970 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083081 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.083000 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.083096 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.083142 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16174b3f-7b3a-4841-9f09-34edcb19934d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.083181 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.083267 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.083210 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.183902 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.183813 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184060 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184013 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184118 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184076 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184168 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184145 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bh8h\" (UniqueName: \"kubernetes.io/projected/16174b3f-7b3a-4841-9f09-34edcb19934d-kube-api-access-7bh8h\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184217 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184197 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184269 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184201 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184269 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184263 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184361 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184306 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184361 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184337 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16174b3f-7b3a-4841-9f09-34edcb19934d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184451 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184367 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.184501 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.184479 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.185099 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.185073 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.185460 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.185433 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.185947 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.185919 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16174b3f-7b3a-4841-9f09-34edcb19934d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.186976 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.186954 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.187299 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.187278 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.192287 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.192267 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bh8h\" (UniqueName: \"kubernetes.io/projected/16174b3f-7b3a-4841-9f09-34edcb19934d-kube-api-access-7bh8h\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.192438 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.192419 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16174b3f-7b3a-4841-9f09-34edcb19934d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6\" (UID: \"16174b3f-7b3a-4841-9f09-34edcb19934d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.232765 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.232744 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:33.388029 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.388003 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6"] Jul 28 14:30:33.389436 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:30:33.389413 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16174b3f_7b3a_4841_9f09_34edcb19934d.slice/crio-59256f849f6dc60278f62b90701c5ad00be5421f48bdda7c234e4962967d9df8 WatchSource:0}: Error finding container 59256f849f6dc60278f62b90701c5ad00be5421f48bdda7c234e4962967d9df8: Status 404 returned error can't find the container with id 59256f849f6dc60278f62b90701c5ad00be5421f48bdda7c234e4962967d9df8 Jul 28 14:30:33.567178 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.567143 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg"] Jul 28 14:30:33.571983 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.571962 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.575393 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.575375 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 28 14:30:33.575490 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.575375 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-g8qcm\"" Jul 28 14:30:33.575919 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.575900 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 28 14:30:33.576494 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.576480 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 28 14:30:33.589368 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.589350 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg"] Jul 28 14:30:33.688540 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688509 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.688699 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688578 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.688699 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688604 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.688699 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688631 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbzwz\" (UniqueName: \"kubernetes.io/projected/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-kube-api-access-zbzwz\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.688827 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688700 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.688827 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688730 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.688827 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.688749 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.783912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.783886 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" event={"ID":"16174b3f-7b3a-4841-9f09-34edcb19934d","Type":"ContainerStarted","Data":"59256f849f6dc60278f62b90701c5ad00be5421f48bdda7c234e4962967d9df8"} Jul 28 14:30:33.789278 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789255 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.789374 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789306 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.789374 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789327 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.789483 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789448 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbzwz\" (UniqueName: \"kubernetes.io/projected/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-kube-api-access-zbzwz\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.789541 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789503 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.789620 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789544 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.789620 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.789582 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.790121 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.790098 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.791717 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.791688 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.791832 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.791819 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.792239 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.792216 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.792239 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.792232 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.797707 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.797690 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.797994 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.797977 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbzwz\" (UniqueName: \"kubernetes.io/projected/0b16d15a-cf1a-4ee7-9240-b9d813ba546a-kube-api-access-zbzwz\") pod \"istiod-openshift-gateway-5677ccb49b-74jxg\" (UID: \"0b16d15a-cf1a-4ee7-9240-b9d813ba546a\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:33.882308 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:33.882256 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:34.044187 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:34.044163 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg"] Jul 28 14:30:34.045028 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:30:34.045006 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b16d15a_cf1a_4ee7_9240_b9d813ba546a.slice/crio-9d6a1e53825dca4cb7f67f63193ce72b843afb160c63f32493fdb53df0d700ba WatchSource:0}: Error finding container 9d6a1e53825dca4cb7f67f63193ce72b843afb160c63f32493fdb53df0d700ba: Status 404 returned error can't find the container with id 9d6a1e53825dca4cb7f67f63193ce72b843afb160c63f32493fdb53df0d700ba Jul 28 14:30:34.790022 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:34.789982 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" event={"ID":"0b16d15a-cf1a-4ee7-9240-b9d813ba546a","Type":"ContainerStarted","Data":"9d6a1e53825dca4cb7f67f63193ce72b843afb160c63f32493fdb53df0d700ba"} Jul 28 14:30:48.976830 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:48.976790 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 14:30:48.977172 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:48.976858 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 14:30:48.977172 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:48.976856 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 14:30:48.977523 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:48.977492 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 14:30:48.977662 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:48.977638 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 14:30:49.867231 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:49.867191 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" event={"ID":"0b16d15a-cf1a-4ee7-9240-b9d813ba546a","Type":"ContainerStarted","Data":"9ed231d0241476e45d453d1cad88490c339dee984475f3a506cb92879dd53e5b"} Jul 28 14:30:49.867437 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:49.867296 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:30:49.868657 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:49.868635 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" event={"ID":"16174b3f-7b3a-4841-9f09-34edcb19934d","Type":"ContainerStarted","Data":"d8686cad35513760147928dff6ded3ab5b3f9203df9fa2adc9ed947f30d479e5"} Jul 28 14:30:49.888827 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:49.888783 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" podStartSLOduration=1.959393795 podStartE2EDuration="16.888768384s" totalImagesPullingTime="14.929374589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:49.885883852 +0000 UTC m=+480.686761407" watchObservedRunningTime="2026-07-28 14:30:49.888768384 +0000 UTC m=+480.689645939" Jul 28 14:30:49.907019 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:49.906979 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" podStartSLOduration=2.321698637 podStartE2EDuration="17.906967116s" totalImagesPullingTime="15.585268479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:30:49.905606111 +0000 UTC m=+480.706483667" watchObservedRunningTime="2026-07-28 14:30:49.906967116 +0000 UTC m=+480.707844672" Jul 28 14:30:50.233788 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:50.233697 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:50.238875 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:50.238849 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:50.873806 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:50.873767 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:50.874685 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:50.874659 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6" Jul 28 14:30:50.875656 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:30:50.875633 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-74jxg" Jul 28 14:31:29.320427 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.320376 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f6686b5f6-5xpqt"] Jul 28 14:31:29.330009 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.329972 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.335033 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.334997 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6686b5f6-5xpqt"] Jul 28 14:31:29.424795 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.424741 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-config\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.424795 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.424796 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8q7c\" (UniqueName: \"kubernetes.io/projected/ed0395d3-4932-4dc1-b345-5c231f384eaa-kube-api-access-c8q7c\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.425052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.424829 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-serving-cert\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.425052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.424863 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-service-ca\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.425052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.424960 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-oauth-config\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.425052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.424999 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-oauth-serving-cert\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.425052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.425033 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-trusted-ca-bundle\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526138 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526090 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-config\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526138 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526137 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8q7c\" (UniqueName: \"kubernetes.io/projected/ed0395d3-4932-4dc1-b345-5c231f384eaa-kube-api-access-c8q7c\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526402 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526264 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-serving-cert\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526402 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526338 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-service-ca\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526528 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526434 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-oauth-config\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526528 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526464 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-oauth-serving-cert\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.526528 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.526512 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-trusted-ca-bundle\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.527074 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.527039 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-config\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.527200 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.527131 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-service-ca\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.527263 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.527204 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-oauth-serving-cert\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.527475 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.527448 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed0395d3-4932-4dc1-b345-5c231f384eaa-trusted-ca-bundle\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.529329 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.529295 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-oauth-config\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.529491 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.529400 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0395d3-4932-4dc1-b345-5c231f384eaa-console-serving-cert\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.535993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.535959 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8q7c\" (UniqueName: \"kubernetes.io/projected/ed0395d3-4932-4dc1-b345-5c231f384eaa-kube-api-access-c8q7c\") pod \"console-7f6686b5f6-5xpqt\" (UID: \"ed0395d3-4932-4dc1-b345-5c231f384eaa\") " pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.645700 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.645596 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:29.810940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:29.810905 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f6686b5f6-5xpqt"] Jul 28 14:31:29.813130 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:31:29.813096 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0395d3_4932_4dc1_b345_5c231f384eaa.slice/crio-126c15c7469c5fe2ada39696a6025eaa909138838bd7db08f0d7a908ad183f4b WatchSource:0}: Error finding container 126c15c7469c5fe2ada39696a6025eaa909138838bd7db08f0d7a908ad183f4b: Status 404 returned error can't find the container with id 126c15c7469c5fe2ada39696a6025eaa909138838bd7db08f0d7a908ad183f4b Jul 28 14:31:30.098969 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:30.098928 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6686b5f6-5xpqt" event={"ID":"ed0395d3-4932-4dc1-b345-5c231f384eaa","Type":"ContainerStarted","Data":"b3447ac577d510d440e7bd0d981c5dc1222e051e547cfbcf15e72db9b759b51d"} Jul 28 14:31:30.099135 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:30.098984 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f6686b5f6-5xpqt" event={"ID":"ed0395d3-4932-4dc1-b345-5c231f384eaa","Type":"ContainerStarted","Data":"126c15c7469c5fe2ada39696a6025eaa909138838bd7db08f0d7a908ad183f4b"} Jul 28 14:31:30.118787 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:30.118719 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f6686b5f6-5xpqt" podStartSLOduration=1.118696427 podStartE2EDuration="1.118696427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:31:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:31:30.117390966 +0000 UTC m=+520.918268523" watchObservedRunningTime="2026-07-28 14:31:30.118696427 +0000 UTC m=+520.919574067" Jul 28 14:31:39.646603 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:39.646527 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:39.647084 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:39.646665 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:39.652121 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:39.652083 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:40.161382 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:40.161348 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f6686b5f6-5xpqt" Jul 28 14:31:40.211904 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:40.211863 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b85f648d5-hcrv5"] Jul 28 14:31:41.409162 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.409060 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss"] Jul 28 14:31:41.412932 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.412908 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.415659 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.415629 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 28 14:31:41.415849 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.415667 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7hsxh\"" Jul 28 14:31:41.416330 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.416308 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 28 14:31:41.420934 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.420890 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss"] Jul 28 14:31:41.546207 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.546156 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g9nw\" (UniqueName: \"kubernetes.io/projected/10ffdfab-cdaa-48b8-85b0-f5947fe34917-kube-api-access-8g9nw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.546406 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.546286 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.546406 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.546323 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.647916 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.647875 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g9nw\" (UniqueName: \"kubernetes.io/projected/10ffdfab-cdaa-48b8-85b0-f5947fe34917-kube-api-access-8g9nw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.648122 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.647966 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.648122 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.647983 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.648398 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.648380 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.648437 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.648412 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.657270 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.657212 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g9nw\" (UniqueName: \"kubernetes.io/projected/10ffdfab-cdaa-48b8-85b0-f5947fe34917-kube-api-access-8g9nw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.727060 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.726959 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:41.887391 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:41.887344 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss"] Jul 28 14:31:41.891083 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:31:41.891041 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ffdfab_cdaa_48b8_85b0_f5947fe34917.slice/crio-7d661e048c25dbd9da13675b722ecd884a02d2f269568dfa8b25c044d2fdfb5d WatchSource:0}: Error finding container 7d661e048c25dbd9da13675b722ecd884a02d2f269568dfa8b25c044d2fdfb5d: Status 404 returned error can't find the container with id 7d661e048c25dbd9da13675b722ecd884a02d2f269568dfa8b25c044d2fdfb5d Jul 28 14:31:42.004856 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.004812 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r"] Jul 28 14:31:42.007848 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.007822 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.018792 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.018753 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r"] Jul 28 14:31:42.152683 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.152640 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7z6n\" (UniqueName: \"kubernetes.io/projected/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-kube-api-access-t7z6n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.152882 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.152809 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.152882 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.152855 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.179161 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.179100 2586 generic.go:356] "Generic (PLEG): container finished" podID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerID="61bdb966f7a563229a2f081a512dcc055302ed2f97f11c127111a15ee53ab054" exitCode=0 Jul 28 14:31:42.179355 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.179188 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" event={"ID":"10ffdfab-cdaa-48b8-85b0-f5947fe34917","Type":"ContainerDied","Data":"61bdb966f7a563229a2f081a512dcc055302ed2f97f11c127111a15ee53ab054"} Jul 28 14:31:42.179406 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.179378 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" event={"ID":"10ffdfab-cdaa-48b8-85b0-f5947fe34917","Type":"ContainerStarted","Data":"7d661e048c25dbd9da13675b722ecd884a02d2f269568dfa8b25c044d2fdfb5d"} Jul 28 14:31:42.254038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.253935 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.254038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.253984 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.254038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.254031 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7z6n\" (UniqueName: \"kubernetes.io/projected/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-kube-api-access-t7z6n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.254523 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.254487 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.254637 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.254526 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.262807 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.262768 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7z6n\" (UniqueName: \"kubernetes.io/projected/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-kube-api-access-t7z6n\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.321033 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.320997 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:42.410185 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.410130 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w"] Jul 28 14:31:42.414658 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.414627 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.424093 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.423987 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w"] Jul 28 14:31:42.488803 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.488767 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r"] Jul 28 14:31:42.490837 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:31:42.490803 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf309ddcd_2cdf_45dc_bc32_0701a4e0d282.slice/crio-8a0b1e67d2b2a5a08f6ee20d49ba388b5f07d5af34cf4a8312f4e8831d3df545 WatchSource:0}: Error finding container 8a0b1e67d2b2a5a08f6ee20d49ba388b5f07d5af34cf4a8312f4e8831d3df545: Status 404 returned error can't find the container with id 8a0b1e67d2b2a5a08f6ee20d49ba388b5f07d5af34cf4a8312f4e8831d3df545 Jul 28 14:31:42.557881 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.557844 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.558101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.557904 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.558101 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.558067 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h77df\" (UniqueName: \"kubernetes.io/projected/e4578db9-400c-4e01-93ef-4e11b2ecfb71-kube-api-access-h77df\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.659279 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.659241 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.659454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.659303 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.659454 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.659366 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h77df\" (UniqueName: \"kubernetes.io/projected/e4578db9-400c-4e01-93ef-4e11b2ecfb71-kube-api-access-h77df\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.659831 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.659795 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.659876 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.659807 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.668866 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.668809 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77df\" (UniqueName: \"kubernetes.io/projected/e4578db9-400c-4e01-93ef-4e11b2ecfb71-kube-api-access-h77df\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.729160 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.729112 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:42.815378 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.815329 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw"] Jul 28 14:31:42.819487 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.819447 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:42.827178 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.827135 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw"] Jul 28 14:31:42.895504 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.895472 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w"] Jul 28 14:31:42.897863 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:31:42.897828 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4578db9_400c_4e01_93ef_4e11b2ecfb71.slice/crio-6e748baba999f35f884083e8d99119ebb4bf1956f42ae1ce5457f33485489a80 WatchSource:0}: Error finding container 6e748baba999f35f884083e8d99119ebb4bf1956f42ae1ce5457f33485489a80: Status 404 returned error can't find the container with id 6e748baba999f35f884083e8d99119ebb4bf1956f42ae1ce5457f33485489a80 Jul 28 14:31:42.962433 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.962375 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:42.962433 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.962438 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptngj\" (UniqueName: \"kubernetes.io/projected/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-kube-api-access-ptngj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:42.962728 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:42.962679 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.063977 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.063932 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptngj\" (UniqueName: \"kubernetes.io/projected/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-kube-api-access-ptngj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.064176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.064057 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.064176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.064099 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.064469 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.064452 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.064529 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.064508 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.073480 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.073432 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptngj\" (UniqueName: \"kubernetes.io/projected/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-kube-api-access-ptngj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.133188 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.133139 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:43.189332 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.189276 2586 generic.go:356] "Generic (PLEG): container finished" podID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerID="afd73cf0df5eecd2400bf80892b876db0a4ad7d01e936befce8c47e707d0cba5" exitCode=0 Jul 28 14:31:43.189534 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.189453 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" event={"ID":"f309ddcd-2cdf-45dc-bc32-0701a4e0d282","Type":"ContainerDied","Data":"afd73cf0df5eecd2400bf80892b876db0a4ad7d01e936befce8c47e707d0cba5"} Jul 28 14:31:43.189534 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.189512 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" event={"ID":"f309ddcd-2cdf-45dc-bc32-0701a4e0d282","Type":"ContainerStarted","Data":"8a0b1e67d2b2a5a08f6ee20d49ba388b5f07d5af34cf4a8312f4e8831d3df545"} Jul 28 14:31:43.195083 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.195039 2586 generic.go:356] "Generic (PLEG): container finished" podID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerID="69cf8aa5cadbe0ca787136a24048e363546bd16de2bf95d580f37c77bf502175" exitCode=0 Jul 28 14:31:43.195596 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.195152 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" event={"ID":"e4578db9-400c-4e01-93ef-4e11b2ecfb71","Type":"ContainerDied","Data":"69cf8aa5cadbe0ca787136a24048e363546bd16de2bf95d580f37c77bf502175"} Jul 28 14:31:43.195596 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.195204 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" event={"ID":"e4578db9-400c-4e01-93ef-4e11b2ecfb71","Type":"ContainerStarted","Data":"6e748baba999f35f884083e8d99119ebb4bf1956f42ae1ce5457f33485489a80"} Jul 28 14:31:43.303894 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:43.303854 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw"] Jul 28 14:31:43.380292 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:31:43.380198 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9a001b_91a5_46d1_b90c_2d5cbee6f5a4.slice/crio-b1b97793d944c3598fe85288e3105e6d7085f55e2bb8c5fecc5160403ed32ac8 WatchSource:0}: Error finding container b1b97793d944c3598fe85288e3105e6d7085f55e2bb8c5fecc5160403ed32ac8: Status 404 returned error can't find the container with id b1b97793d944c3598fe85288e3105e6d7085f55e2bb8c5fecc5160403ed32ac8 Jul 28 14:31:44.203198 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:44.203162 2586 generic.go:356] "Generic (PLEG): container finished" podID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerID="758ee729176e3d3db4972c2f11c4db6d49054df0b084e8f09763ea3eb99428ef" exitCode=0 Jul 28 14:31:44.203940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:44.203258 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" event={"ID":"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4","Type":"ContainerDied","Data":"758ee729176e3d3db4972c2f11c4db6d49054df0b084e8f09763ea3eb99428ef"} Jul 28 14:31:44.203940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:44.203331 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" event={"ID":"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4","Type":"ContainerStarted","Data":"b1b97793d944c3598fe85288e3105e6d7085f55e2bb8c5fecc5160403ed32ac8"} Jul 28 14:31:44.205979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:44.205939 2586 generic.go:356] "Generic (PLEG): container finished" podID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerID="a83144dfd4ecbbd3baa7ae505dbbcb38d40f65e23f9af3ad9515b0bcacb2ded6" exitCode=0 Jul 28 14:31:44.206183 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:44.205999 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" event={"ID":"10ffdfab-cdaa-48b8-85b0-f5947fe34917","Type":"ContainerDied","Data":"a83144dfd4ecbbd3baa7ae505dbbcb38d40f65e23f9af3ad9515b0bcacb2ded6"} Jul 28 14:31:45.214849 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:45.214804 2586 generic.go:356] "Generic (PLEG): container finished" podID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerID="92b54eeaab23cd443aa66c7ca9326d52752b9c4a842f0ecce9548f367ecd2c14" exitCode=0 Jul 28 14:31:45.215373 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:45.214902 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" event={"ID":"f309ddcd-2cdf-45dc-bc32-0701a4e0d282","Type":"ContainerDied","Data":"92b54eeaab23cd443aa66c7ca9326d52752b9c4a842f0ecce9548f367ecd2c14"} Jul 28 14:31:45.217480 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:45.217451 2586 generic.go:356] "Generic (PLEG): container finished" podID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerID="733c3d0b087ba3a475a0b934b14a2d5f8ada5ff2c94fbc79acba3bdee4ed4572" exitCode=0 Jul 28 14:31:45.217603 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:45.217560 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" event={"ID":"e4578db9-400c-4e01-93ef-4e11b2ecfb71","Type":"ContainerDied","Data":"733c3d0b087ba3a475a0b934b14a2d5f8ada5ff2c94fbc79acba3bdee4ed4572"} Jul 28 14:31:45.220878 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:45.220749 2586 generic.go:356] "Generic (PLEG): container finished" podID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerID="af500e9a6a2aa263ed07cdf6a87b01944fe16000f407043614b72869b8d77620" exitCode=0 Jul 28 14:31:45.220878 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:45.220793 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" event={"ID":"10ffdfab-cdaa-48b8-85b0-f5947fe34917","Type":"ContainerDied","Data":"af500e9a6a2aa263ed07cdf6a87b01944fe16000f407043614b72869b8d77620"} Jul 28 14:31:46.231176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.231130 2586 generic.go:356] "Generic (PLEG): container finished" podID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerID="ec531fa14116f2df7bbec30ebcf644632f4d734d9258c2cc0ee8a530c5b0c1e3" exitCode=0 Jul 28 14:31:46.231675 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.231233 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" event={"ID":"f309ddcd-2cdf-45dc-bc32-0701a4e0d282","Type":"ContainerDied","Data":"ec531fa14116f2df7bbec30ebcf644632f4d734d9258c2cc0ee8a530c5b0c1e3"} Jul 28 14:31:46.233876 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.233835 2586 generic.go:356] "Generic (PLEG): container finished" podID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerID="2c4886eda90453dc7e9c5918bb1ec216384c5015f0bb203f2fabd73bb0a89fb8" exitCode=0 Jul 28 14:31:46.234046 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.233926 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" event={"ID":"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4","Type":"ContainerDied","Data":"2c4886eda90453dc7e9c5918bb1ec216384c5015f0bb203f2fabd73bb0a89fb8"} Jul 28 14:31:46.236980 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.236937 2586 generic.go:356] "Generic (PLEG): container finished" podID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerID="d712b236c9139d12e3dfd3f32d04af95d70e279911df831e97c6d21a9552fd16" exitCode=0 Jul 28 14:31:46.237181 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.236985 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" event={"ID":"e4578db9-400c-4e01-93ef-4e11b2ecfb71","Type":"ContainerDied","Data":"d712b236c9139d12e3dfd3f32d04af95d70e279911df831e97c6d21a9552fd16"} Jul 28 14:31:46.401280 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.401247 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:46.503036 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.502928 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-util\") pod \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " Jul 28 14:31:46.503036 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.503030 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g9nw\" (UniqueName: \"kubernetes.io/projected/10ffdfab-cdaa-48b8-85b0-f5947fe34917-kube-api-access-8g9nw\") pod \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " Jul 28 14:31:46.503260 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.503050 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-bundle\") pod \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\" (UID: \"10ffdfab-cdaa-48b8-85b0-f5947fe34917\") " Jul 28 14:31:46.503597 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.503536 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-bundle" (OuterVolumeSpecName: "bundle") pod "10ffdfab-cdaa-48b8-85b0-f5947fe34917" (UID: "10ffdfab-cdaa-48b8-85b0-f5947fe34917"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:46.505711 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.505673 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ffdfab-cdaa-48b8-85b0-f5947fe34917-kube-api-access-8g9nw" (OuterVolumeSpecName: "kube-api-access-8g9nw") pod "10ffdfab-cdaa-48b8-85b0-f5947fe34917" (UID: "10ffdfab-cdaa-48b8-85b0-f5947fe34917"). InnerVolumeSpecName "kube-api-access-8g9nw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:31:46.509395 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.509353 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-util" (OuterVolumeSpecName: "util") pod "10ffdfab-cdaa-48b8-85b0-f5947fe34917" (UID: "10ffdfab-cdaa-48b8-85b0-f5947fe34917"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:46.605024 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.604969 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:46.605024 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.605007 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g9nw\" (UniqueName: \"kubernetes.io/projected/10ffdfab-cdaa-48b8-85b0-f5947fe34917-kube-api-access-8g9nw\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:46.605024 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:46.605018 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ffdfab-cdaa-48b8-85b0-f5947fe34917-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:47.246880 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.246842 2586 generic.go:356] "Generic (PLEG): container finished" podID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerID="0a8c3a0d85ff4ff978e2ec2fa577c2d3b32c087a459958cdca696cde3596a5e9" exitCode=0 Jul 28 14:31:47.247309 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.246933 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" event={"ID":"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4","Type":"ContainerDied","Data":"0a8c3a0d85ff4ff978e2ec2fa577c2d3b32c087a459958cdca696cde3596a5e9"} Jul 28 14:31:47.249819 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.249778 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" event={"ID":"10ffdfab-cdaa-48b8-85b0-f5947fe34917","Type":"ContainerDied","Data":"7d661e048c25dbd9da13675b722ecd884a02d2f269568dfa8b25c044d2fdfb5d"} Jul 28 14:31:47.250013 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.249829 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss" Jul 28 14:31:47.250013 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.249847 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d661e048c25dbd9da13675b722ecd884a02d2f269568dfa8b25c044d2fdfb5d" Jul 28 14:31:47.437427 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.437382 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:47.441785 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.441747 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:47.515032 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.514916 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-util\") pod \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " Jul 28 14:31:47.515032 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.514963 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-bundle\") pod \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " Jul 28 14:31:47.515032 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.515012 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h77df\" (UniqueName: \"kubernetes.io/projected/e4578db9-400c-4e01-93ef-4e11b2ecfb71-kube-api-access-h77df\") pod \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " Jul 28 14:31:47.515032 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.515033 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-bundle\") pod \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " Jul 28 14:31:47.515388 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.515091 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7z6n\" (UniqueName: \"kubernetes.io/projected/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-kube-api-access-t7z6n\") pod \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\" (UID: \"f309ddcd-2cdf-45dc-bc32-0701a4e0d282\") " Jul 28 14:31:47.515388 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.515160 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-util\") pod \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\" (UID: \"e4578db9-400c-4e01-93ef-4e11b2ecfb71\") " Jul 28 14:31:47.516003 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.515864 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-bundle" (OuterVolumeSpecName: "bundle") pod "e4578db9-400c-4e01-93ef-4e11b2ecfb71" (UID: "e4578db9-400c-4e01-93ef-4e11b2ecfb71"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:47.516003 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.515952 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-bundle" (OuterVolumeSpecName: "bundle") pod "f309ddcd-2cdf-45dc-bc32-0701a4e0d282" (UID: "f309ddcd-2cdf-45dc-bc32-0701a4e0d282"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:47.517680 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.517640 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4578db9-400c-4e01-93ef-4e11b2ecfb71-kube-api-access-h77df" (OuterVolumeSpecName: "kube-api-access-h77df") pod "e4578db9-400c-4e01-93ef-4e11b2ecfb71" (UID: "e4578db9-400c-4e01-93ef-4e11b2ecfb71"). InnerVolumeSpecName "kube-api-access-h77df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:31:47.517839 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.517808 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-kube-api-access-t7z6n" (OuterVolumeSpecName: "kube-api-access-t7z6n") pod "f309ddcd-2cdf-45dc-bc32-0701a4e0d282" (UID: "f309ddcd-2cdf-45dc-bc32-0701a4e0d282"). InnerVolumeSpecName "kube-api-access-t7z6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:31:47.521734 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.521686 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-util" (OuterVolumeSpecName: "util") pod "f309ddcd-2cdf-45dc-bc32-0701a4e0d282" (UID: "f309ddcd-2cdf-45dc-bc32-0701a4e0d282"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:47.521734 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.521700 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-util" (OuterVolumeSpecName: "util") pod "e4578db9-400c-4e01-93ef-4e11b2ecfb71" (UID: "e4578db9-400c-4e01-93ef-4e11b2ecfb71"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:47.616328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.616285 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:47.616328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.616321 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:47.616328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.616329 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:47.616328 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.616338 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h77df\" (UniqueName: \"kubernetes.io/projected/e4578db9-400c-4e01-93ef-4e11b2ecfb71-kube-api-access-h77df\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:47.616711 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.616347 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4578db9-400c-4e01-93ef-4e11b2ecfb71-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:47.616711 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:47.616356 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7z6n\" (UniqueName: \"kubernetes.io/projected/f309ddcd-2cdf-45dc-bc32-0701a4e0d282-kube-api-access-t7z6n\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:48.259231 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.259187 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" Jul 28 14:31:48.259231 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.259206 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r" event={"ID":"f309ddcd-2cdf-45dc-bc32-0701a4e0d282","Type":"ContainerDied","Data":"8a0b1e67d2b2a5a08f6ee20d49ba388b5f07d5af34cf4a8312f4e8831d3df545"} Jul 28 14:31:48.259765 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.259261 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a0b1e67d2b2a5a08f6ee20d49ba388b5f07d5af34cf4a8312f4e8831d3df545" Jul 28 14:31:48.262375 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.262331 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" event={"ID":"e4578db9-400c-4e01-93ef-4e11b2ecfb71","Type":"ContainerDied","Data":"6e748baba999f35f884083e8d99119ebb4bf1956f42ae1ce5457f33485489a80"} Jul 28 14:31:48.262375 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.262373 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w" Jul 28 14:31:48.262653 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.262395 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e748baba999f35f884083e8d99119ebb4bf1956f42ae1ce5457f33485489a80" Jul 28 14:31:48.414294 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.414249 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:48.526526 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.526404 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptngj\" (UniqueName: \"kubernetes.io/projected/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-kube-api-access-ptngj\") pod \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " Jul 28 14:31:48.526759 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.526544 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-bundle\") pod \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " Jul 28 14:31:48.526759 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.526597 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-util\") pod \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\" (UID: \"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4\") " Jul 28 14:31:48.527119 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.527073 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-bundle" (OuterVolumeSpecName: "bundle") pod "fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" (UID: "fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:48.529004 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.528957 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-kube-api-access-ptngj" (OuterVolumeSpecName: "kube-api-access-ptngj") pod "fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" (UID: "fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4"). InnerVolumeSpecName "kube-api-access-ptngj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:31:48.532497 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.532435 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-util" (OuterVolumeSpecName: "util") pod "fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" (UID: "fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 14:31:48.628127 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.628082 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:48.628127 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.628117 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-util\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:48.628127 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:48.628130 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptngj\" (UniqueName: \"kubernetes.io/projected/fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4-kube-api-access-ptngj\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:31:49.270706 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:49.270670 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" Jul 28 14:31:49.270706 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:49.270696 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw" event={"ID":"fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4","Type":"ContainerDied","Data":"b1b97793d944c3598fe85288e3105e6d7085f55e2bb8c5fecc5160403ed32ac8"} Jul 28 14:31:49.271203 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:31:49.270783 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1b97793d944c3598fe85288e3105e6d7085f55e2bb8c5fecc5160403ed32ac8" Jul 28 14:32:05.233253 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.233183 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b85f648d5-hcrv5" podUID="6d470205-91ef-49c4-a40e-fc5a2d131ddb" containerName="console" containerID="cri-o://2436908fb249884aa6fed5a59086f1026188d92cef58b010dacd322c355a9492" gracePeriod=15 Jul 28 14:32:05.365121 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.365090 2586 generic.go:356] "Generic (PLEG): container finished" podID="6d470205-91ef-49c4-a40e-fc5a2d131ddb" containerID="2436908fb249884aa6fed5a59086f1026188d92cef58b010dacd322c355a9492" exitCode=0 Jul 28 14:32:05.365303 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.365159 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85f648d5-hcrv5" event={"ID":"6d470205-91ef-49c4-a40e-fc5a2d131ddb","Type":"ContainerDied","Data":"2436908fb249884aa6fed5a59086f1026188d92cef58b010dacd322c355a9492"} Jul 28 14:32:05.500068 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.499991 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:32:05.575089 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575049 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cvb7\" (UniqueName: \"kubernetes.io/projected/6d470205-91ef-49c4-a40e-fc5a2d131ddb-kube-api-access-8cvb7\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575299 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575096 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-serving-cert\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575299 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575142 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-trusted-ca-bundle\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575419 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575311 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-oauth-config\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575419 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575380 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-oauth-serving-cert\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575533 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575424 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-service-ca\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575533 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575490 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-config\") pod \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\" (UID: \"6d470205-91ef-49c4-a40e-fc5a2d131ddb\") " Jul 28 14:32:05.575669 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575618 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:32:05.575947 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575920 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-trusted-ca-bundle\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:05.575947 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575937 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-config" (OuterVolumeSpecName: "console-config") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:32:05.576108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575957 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:32:05.576108 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.575997 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:32:05.577481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.577436 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:32:05.577677 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.577477 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d470205-91ef-49c4-a40e-fc5a2d131ddb-kube-api-access-8cvb7" (OuterVolumeSpecName: "kube-api-access-8cvb7") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "kube-api-access-8cvb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:32:05.577761 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.577736 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d470205-91ef-49c4-a40e-fc5a2d131ddb" (UID: "6d470205-91ef-49c4-a40e-fc5a2d131ddb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:32:05.676369 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.676325 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:05.676369 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.676360 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cvb7\" (UniqueName: \"kubernetes.io/projected/6d470205-91ef-49c4-a40e-fc5a2d131ddb-kube-api-access-8cvb7\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:05.676369 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.676371 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:05.676369 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.676380 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d470205-91ef-49c4-a40e-fc5a2d131ddb-console-oauth-config\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:05.676677 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.676391 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-oauth-serving-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:05.676677 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:05.676399 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d470205-91ef-49c4-a40e-fc5a2d131ddb-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:32:06.372022 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:06.371989 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85f648d5-hcrv5" Jul 28 14:32:06.372022 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:06.372014 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85f648d5-hcrv5" event={"ID":"6d470205-91ef-49c4-a40e-fc5a2d131ddb","Type":"ContainerDied","Data":"b0029b64cef54e5c8296114346bba2437adbb08a023fad0f13f18d9ed7ee55ab"} Jul 28 14:32:06.372504 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:06.372094 2586 scope.go:117] "RemoveContainer" containerID="2436908fb249884aa6fed5a59086f1026188d92cef58b010dacd322c355a9492" Jul 28 14:32:06.391240 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:06.391210 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b85f648d5-hcrv5"] Jul 28 14:32:06.396951 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:06.396930 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b85f648d5-hcrv5"] Jul 28 14:32:07.691599 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:32:07.691546 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d470205-91ef-49c4-a40e-fc5a2d131ddb" path="/var/lib/kubelet/pods/6d470205-91ef-49c4-a40e-fc5a2d131ddb/volumes" Jul 28 14:33:51.453061 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453024 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f564445cb-wzm65"] Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453392 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453405 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453438 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453443 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453458 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453463 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453469 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453474 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="util" Jul 28 14:33:51.453478 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453481 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453486 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453493 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453498 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453504 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453509 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453518 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d470205-91ef-49c4-a40e-fc5a2d131ddb" containerName="console" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453524 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d470205-91ef-49c4-a40e-fc5a2d131ddb" containerName="console" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453530 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453535 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453540 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453545 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453571 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453576 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453583 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453588 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453593 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453598 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="pull" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453647 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f309ddcd-2cdf-45dc-bc32-0701a4e0d282" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453658 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d470205-91ef-49c4-a40e-fc5a2d131ddb" containerName="console" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453664 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4578db9-400c-4e01-93ef-4e11b2ecfb71" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453669 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4" containerName="extract" Jul 28 14:33:51.453768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.453674 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ffdfab-cdaa-48b8-85b0-f5947fe34917" containerName="extract" Jul 28 14:33:51.456254 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.456240 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.458927 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.458901 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 28 14:33:51.459243 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.459228 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h752g\"" Jul 28 14:33:51.459243 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.459231 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Jul 28 14:33:51.459399 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.459286 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 28 14:33:51.465979 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.465951 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f564445cb-wzm65"] Jul 28 14:33:51.511683 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.511654 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.511851 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.511770 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-metrics-certs\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.511851 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.511799 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbl7k\" (UniqueName: \"kubernetes.io/projected/e1f0df56-b62e-4222-869b-1fee717269f6-kube-api-access-rbl7k\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.612670 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.612639 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-metrics-certs\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.612830 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.612677 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbl7k\" (UniqueName: \"kubernetes.io/projected/e1f0df56-b62e-4222-869b-1fee717269f6-kube-api-access-rbl7k\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.612830 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.612739 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.612902 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:33:51.612878 2586 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 28 14:33:51.612967 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:33:51.612955 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert podName:e1f0df56-b62e-4222-869b-1fee717269f6 nodeName:}" failed. No retries permitted until 2026-07-28 14:33:52.112932278 +0000 UTC m=+662.913809826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert") pod "maas-controller-f564445cb-wzm65" (UID: "e1f0df56-b62e-4222-869b-1fee717269f6") : secret "maas-controller-webhook-cert" not found Jul 28 14:33:51.615018 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.614989 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-metrics-certs\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:51.624710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:51.624689 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbl7k\" (UniqueName: \"kubernetes.io/projected/e1f0df56-b62e-4222-869b-1fee717269f6-kube-api-access-rbl7k\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:52.117964 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:52.117932 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:52.118141 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:33:52.118077 2586 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 28 14:33:52.118188 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:33:52.118143 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert podName:e1f0df56-b62e-4222-869b-1fee717269f6 nodeName:}" failed. No retries permitted until 2026-07-28 14:33:53.118125212 +0000 UTC m=+663.919002747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert") pod "maas-controller-f564445cb-wzm65" (UID: "e1f0df56-b62e-4222-869b-1fee717269f6") : secret "maas-controller-webhook-cert" not found Jul 28 14:33:53.127057 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:53.127023 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:53.129318 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:53.129299 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") pod \"maas-controller-f564445cb-wzm65\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:53.268137 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:53.268098 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:53.406601 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:53.401571 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f564445cb-wzm65"] Jul 28 14:33:53.939730 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:53.939696 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f564445cb-wzm65" event={"ID":"e1f0df56-b62e-4222-869b-1fee717269f6","Type":"ContainerStarted","Data":"2b2827db7af22d6736e64af68b45f8b44acb033f7d4bc9544115b3a721909464"} Jul 28 14:33:56.962561 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:56.962520 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f564445cb-wzm65" event={"ID":"e1f0df56-b62e-4222-869b-1fee717269f6","Type":"ContainerStarted","Data":"49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8"} Jul 28 14:33:56.962956 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:56.962619 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:33:56.982318 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:33:56.982270 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f564445cb-wzm65" podStartSLOduration=3.155009084 podStartE2EDuration="5.982249143s" totalImagesPullingTime="2.827240059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:33:56.98167542 +0000 UTC m=+667.782552977" watchObservedRunningTime="2026-07-28 14:33:56.982249143 +0000 UTC m=+667.783126698" Jul 28 14:34:06.689800 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:06.689764 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f564445cb-wzm65"] Jul 28 14:34:06.690344 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:06.689946 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f564445cb-wzm65" podUID="e1f0df56-b62e-4222-869b-1fee717269f6" containerName="manager" containerID="cri-o://49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8" gracePeriod=30 Jul 28 14:34:06.691528 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:06.691502 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:34:06.944060 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:06.943997 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:34:07.014457 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.014413 2586 generic.go:356] "Generic (PLEG): container finished" podID="e1f0df56-b62e-4222-869b-1fee717269f6" containerID="49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8" exitCode=0 Jul 28 14:34:07.014643 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.014476 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f564445cb-wzm65" Jul 28 14:34:07.014643 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.014502 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f564445cb-wzm65" event={"ID":"e1f0df56-b62e-4222-869b-1fee717269f6","Type":"ContainerDied","Data":"49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8"} Jul 28 14:34:07.014643 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.014602 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f564445cb-wzm65" event={"ID":"e1f0df56-b62e-4222-869b-1fee717269f6","Type":"ContainerDied","Data":"2b2827db7af22d6736e64af68b45f8b44acb033f7d4bc9544115b3a721909464"} Jul 28 14:34:07.014643 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.014632 2586 scope.go:117] "RemoveContainer" containerID="49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8" Jul 28 14:34:07.023113 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.023099 2586 scope.go:117] "RemoveContainer" containerID="49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8" Jul 28 14:34:07.023376 ip-10-0-131-88 kubenswrapper[2586]: E0728 14:34:07.023350 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8\": container with ID starting with 49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8 not found: ID does not exist" containerID="49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8" Jul 28 14:34:07.023485 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.023386 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8"} err="failed to get container status \"49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8\": rpc error: code = NotFound desc = could not find container \"49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8\": container with ID starting with 49ce0939cad6edd53281b043cb742d1d976c8d98980dee9103497f13b8ceeee8 not found: ID does not exist" Jul 28 14:34:07.064371 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.064352 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-metrics-certs\") pod \"e1f0df56-b62e-4222-869b-1fee717269f6\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " Jul 28 14:34:07.064481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.064383 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") pod \"e1f0df56-b62e-4222-869b-1fee717269f6\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " Jul 28 14:34:07.064481 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.064426 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbl7k\" (UniqueName: \"kubernetes.io/projected/e1f0df56-b62e-4222-869b-1fee717269f6-kube-api-access-rbl7k\") pod \"e1f0df56-b62e-4222-869b-1fee717269f6\" (UID: \"e1f0df56-b62e-4222-869b-1fee717269f6\") " Jul 28 14:34:07.066408 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.066389 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e1f0df56-b62e-4222-869b-1fee717269f6" (UID: "e1f0df56-b62e-4222-869b-1fee717269f6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:34:07.066509 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.066443 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e1f0df56-b62e-4222-869b-1fee717269f6" (UID: "e1f0df56-b62e-4222-869b-1fee717269f6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 14:34:07.066569 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.066504 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1f0df56-b62e-4222-869b-1fee717269f6-kube-api-access-rbl7k" (OuterVolumeSpecName: "kube-api-access-rbl7k") pod "e1f0df56-b62e-4222-869b-1fee717269f6" (UID: "e1f0df56-b62e-4222-869b-1fee717269f6"). InnerVolumeSpecName "kube-api-access-rbl7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 14:34:07.165387 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.165361 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbl7k\" (UniqueName: \"kubernetes.io/projected/e1f0df56-b62e-4222-869b-1fee717269f6-kube-api-access-rbl7k\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:34:07.165387 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.165385 2586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-metrics-certs\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:34:07.165387 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.165395 2586 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1f0df56-b62e-4222-869b-1fee717269f6-webhook-cert\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:34:07.337544 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.337503 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f564445cb-wzm65"] Jul 28 14:34:07.340788 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.340765 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f564445cb-wzm65"] Jul 28 14:34:07.690954 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:07.690874 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1f0df56-b62e-4222-869b-1fee717269f6" path="/var/lib/kubelet/pods/e1f0df56-b62e-4222-869b-1fee717269f6/volumes" Jul 28 14:34:26.152038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.152009 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-569f669744-tlbqv"] Jul 28 14:34:26.152401 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.152374 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1f0df56-b62e-4222-869b-1fee717269f6" containerName="manager" Jul 28 14:34:26.152401 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.152385 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1f0df56-b62e-4222-869b-1fee717269f6" containerName="manager" Jul 28 14:34:26.152473 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.152442 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1f0df56-b62e-4222-869b-1fee717269f6" containerName="manager" Jul 28 14:34:26.155246 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.155231 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.157768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.157747 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 28 14:34:26.157768 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.157762 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 28 14:34:26.157894 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.157763 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 28 14:34:26.167350 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.167328 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-569f669744-tlbqv"] Jul 28 14:34:26.241855 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.241831 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92qq9\" (UniqueName: \"kubernetes.io/projected/cccd086d-f697-4fc1-98bc-dcced5390d70-kube-api-access-92qq9\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.241969 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.241862 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cccd086d-f697-4fc1-98bc-dcced5390d70-openshift-service-ca\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.241969 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.241897 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cccd086d-f697-4fc1-98bc-dcced5390d70-maas-api-tls\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.342721 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.342690 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qq9\" (UniqueName: \"kubernetes.io/projected/cccd086d-f697-4fc1-98bc-dcced5390d70-kube-api-access-92qq9\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.342858 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.342733 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cccd086d-f697-4fc1-98bc-dcced5390d70-openshift-service-ca\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.342858 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.342784 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cccd086d-f697-4fc1-98bc-dcced5390d70-maas-api-tls\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.343353 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.343333 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cccd086d-f697-4fc1-98bc-dcced5390d70-openshift-service-ca\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.345157 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.345139 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cccd086d-f697-4fc1-98bc-dcced5390d70-maas-api-tls\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.351578 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.351526 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92qq9\" (UniqueName: \"kubernetes.io/projected/cccd086d-f697-4fc1-98bc-dcced5390d70-kube-api-access-92qq9\") pod \"maas-api-569f669744-tlbqv\" (UID: \"cccd086d-f697-4fc1-98bc-dcced5390d70\") " pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.465936 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.465874 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:26.627052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.627030 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-569f669744-tlbqv"] Jul 28 14:34:26.628917 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:34:26.628893 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcccd086d_f697_4fc1_98bc_dcced5390d70.slice/crio-9716e55dc17623a45592c4f9635cdf5f70d4950afbfe28434916e4fcf77a9912 WatchSource:0}: Error finding container 9716e55dc17623a45592c4f9635cdf5f70d4950afbfe28434916e4fcf77a9912: Status 404 returned error can't find the container with id 9716e55dc17623a45592c4f9635cdf5f70d4950afbfe28434916e4fcf77a9912 Jul 28 14:34:26.630232 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.630217 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 14:34:26.864014 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.863985 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t"] Jul 28 14:34:26.868313 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.868295 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:26.871362 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.871342 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 28 14:34:26.879186 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.879159 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t"] Jul 28 14:34:26.947188 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.947155 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdsv5\" (UniqueName: \"kubernetes.io/projected/3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4-kube-api-access-sdsv5\") pod \"payload-pre-processing-687dbb6cf6-b8c9t\" (UID: \"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:26.947314 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:26.947274 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-b8c9t\" (UID: \"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:27.047912 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.047878 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdsv5\" (UniqueName: \"kubernetes.io/projected/3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4-kube-api-access-sdsv5\") pod \"payload-pre-processing-687dbb6cf6-b8c9t\" (UID: \"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:27.048059 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.047958 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-b8c9t\" (UID: \"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:27.048641 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.048622 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-b8c9t\" (UID: \"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:27.057004 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.056981 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdsv5\" (UniqueName: \"kubernetes.io/projected/3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4-kube-api-access-sdsv5\") pod \"payload-pre-processing-687dbb6cf6-b8c9t\" (UID: \"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:27.121038 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.120942 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" event={"ID":"cccd086d-f697-4fc1-98bc-dcced5390d70","Type":"ContainerStarted","Data":"9716e55dc17623a45592c4f9635cdf5f70d4950afbfe28434916e4fcf77a9912"} Jul 28 14:34:27.185056 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.185025 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:27.320043 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:27.320016 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t"] Jul 28 14:34:27.321504 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:34:27.321477 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab2eb85_0d1e_4fc6_be67_efb3f38aafd4.slice/crio-5004120bc684d031478e3243fb92b6baaaab04173b725aa7270bf68966eecbea WatchSource:0}: Error finding container 5004120bc684d031478e3243fb92b6baaaab04173b725aa7270bf68966eecbea: Status 404 returned error can't find the container with id 5004120bc684d031478e3243fb92b6baaaab04173b725aa7270bf68966eecbea Jul 28 14:34:28.130326 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:28.130285 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" event={"ID":"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4","Type":"ContainerStarted","Data":"5004120bc684d031478e3243fb92b6baaaab04173b725aa7270bf68966eecbea"} Jul 28 14:34:29.139302 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:29.139257 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" event={"ID":"cccd086d-f697-4fc1-98bc-dcced5390d70","Type":"ContainerStarted","Data":"7b22165bb6a3ee23a962088cc39a154e359a4e24901948a976ed23b7f1242488"} Jul 28 14:34:29.139747 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:29.139516 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:29.155603 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:29.155531 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" podStartSLOduration=0.866231733 podStartE2EDuration="3.155514007s" totalImagesPullingTime="2.289282274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:34:29.153508485 +0000 UTC m=+699.954386040" watchObservedRunningTime="2026-07-28 14:34:29.155514007 +0000 UTC m=+699.956391597" Jul 28 14:34:30.145988 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:30.145953 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" event={"ID":"3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4","Type":"ContainerStarted","Data":"42e39ba5ade23951cd333060ed7df192e5ec84b982fe347f22098b7ed1cdcba9"} Jul 28 14:34:30.146357 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:30.146281 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:30.163940 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:30.163899 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" podStartSLOduration=1.876826385 podStartE2EDuration="4.163885599s" totalImagesPullingTime="2.287059214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:34:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:34:30.161039972 +0000 UTC m=+700.961917531" watchObservedRunningTime="2026-07-28 14:34:30.163885599 +0000 UTC m=+700.964763155" Jul 28 14:34:35.151510 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:35.151483 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-569f669744-tlbqv" Jul 28 14:34:41.154714 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:41.154680 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-b8c9t" Jul 28 14:34:46.400001 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.399964 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d"] Jul 28 14:34:46.402407 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.402392 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.405981 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.405962 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bnhwc\"" Jul 28 14:34:46.405981 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.405975 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 28 14:34:46.406149 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.405999 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 28 14:34:46.406149 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.406011 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 28 14:34:46.413299 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.413281 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d"] Jul 28 14:34:46.534495 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.534465 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.534709 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.534500 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.534709 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.534524 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.534709 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.534674 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.534709 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.534700 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be827ab-615b-4e18-93fd-2d672917c2d1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.534874 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.534729 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngljw\" (UniqueName: \"kubernetes.io/projected/7be827ab-615b-4e18-93fd-2d672917c2d1-kube-api-access-ngljw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.635993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.635953 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.635993 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.635996 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636018 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636067 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636085 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be827ab-615b-4e18-93fd-2d672917c2d1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636257 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636105 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngljw\" (UniqueName: \"kubernetes.io/projected/7be827ab-615b-4e18-93fd-2d672917c2d1-kube-api-access-ngljw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636474 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636436 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636474 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636460 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.636587 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.636511 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.638283 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.638257 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7be827ab-615b-4e18-93fd-2d672917c2d1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.638583 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.638562 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7be827ab-615b-4e18-93fd-2d672917c2d1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.644195 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.644173 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngljw\" (UniqueName: \"kubernetes.io/projected/7be827ab-615b-4e18-93fd-2d672917c2d1-kube-api-access-ngljw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6hz8d\" (UID: \"7be827ab-615b-4e18-93fd-2d672917c2d1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.712689 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.712613 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:46.845063 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:46.845033 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d"] Jul 28 14:34:46.846179 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:34:46.846147 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be827ab_615b_4e18_93fd_2d672917c2d1.slice/crio-7b59b28398fdf6217ed98d22bcac088de7f575d360bd0fb417e511b758e5dc94 WatchSource:0}: Error finding container 7b59b28398fdf6217ed98d22bcac088de7f575d360bd0fb417e511b758e5dc94: Status 404 returned error can't find the container with id 7b59b28398fdf6217ed98d22bcac088de7f575d360bd0fb417e511b758e5dc94 Jul 28 14:34:47.248590 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:47.248536 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" event={"ID":"7be827ab-615b-4e18-93fd-2d672917c2d1","Type":"ContainerStarted","Data":"7b59b28398fdf6217ed98d22bcac088de7f575d360bd0fb417e511b758e5dc94"} Jul 28 14:34:48.257304 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:48.257209 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" event={"ID":"7be827ab-615b-4e18-93fd-2d672917c2d1","Type":"ContainerStarted","Data":"b95ad429ebb9a16c640c9f75572691f6d8946a174180b33884c6d4d376b6f163"} Jul 28 14:34:48.274637 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:48.274589 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" podStartSLOduration=1.2964575680000001 podStartE2EDuration="2.274573067s" totalImagesPullingTime="978.115499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:34:48.27384667 +0000 UTC m=+719.074724225" watchObservedRunningTime="2026-07-28 14:34:48.274573067 +0000 UTC m=+719.075450668" Jul 28 14:34:56.175877 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.175788 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb"] Jul 28 14:34:56.179737 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.179719 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.182173 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.182151 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 28 14:34:56.190714 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.190693 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb"] Jul 28 14:34:56.323739 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.323710 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.323903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.323747 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvzvj\" (UniqueName: \"kubernetes.io/projected/0d6522c1-7dbc-4f25-9e49-8e7380978b71-kube-api-access-nvzvj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.323903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.323770 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.323903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.323795 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6522c1-7dbc-4f25-9e49-8e7380978b71-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.323903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.323815 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.323903 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.323858 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.424539 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424509 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.424710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424575 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvzvj\" (UniqueName: \"kubernetes.io/projected/0d6522c1-7dbc-4f25-9e49-8e7380978b71-kube-api-access-nvzvj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.424710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424599 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.424710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424631 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6522c1-7dbc-4f25-9e49-8e7380978b71-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.424710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424650 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.424710 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424664 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.425014 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.424991 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.425067 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.425050 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.425111 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.425087 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.426775 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.426724 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d6522c1-7dbc-4f25-9e49-8e7380978b71-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.427119 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.427098 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d6522c1-7dbc-4f25-9e49-8e7380978b71-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.438638 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.438609 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvzvj\" (UniqueName: \"kubernetes.io/projected/0d6522c1-7dbc-4f25-9e49-8e7380978b71-kube-api-access-nvzvj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mxppb\" (UID: \"0d6522c1-7dbc-4f25-9e49-8e7380978b71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.491169 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.491145 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:34:56.624602 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.624576 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb"] Jul 28 14:34:56.626120 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:34:56.626095 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d6522c1_7dbc_4f25_9e49_8e7380978b71.slice/crio-b3e3a0c6f8ff42dcb70f40b4301540fa221edc318d24b31729901d435e64a9a1 WatchSource:0}: Error finding container b3e3a0c6f8ff42dcb70f40b4301540fa221edc318d24b31729901d435e64a9a1: Status 404 returned error can't find the container with id b3e3a0c6f8ff42dcb70f40b4301540fa221edc318d24b31729901d435e64a9a1 Jul 28 14:34:56.713222 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.713147 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:56.713222 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.713191 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:56.725745 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:56.725719 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:57.308472 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:57.308432 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" event={"ID":"0d6522c1-7dbc-4f25-9e49-8e7380978b71","Type":"ContainerStarted","Data":"fc07f123efa373c11549bb1bcc5e0b0b8a1e09c806c3b7ae42793a9f2f9682cf"} Jul 28 14:34:57.308923 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:57.308496 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" event={"ID":"0d6522c1-7dbc-4f25-9e49-8e7380978b71","Type":"ContainerStarted","Data":"b3e3a0c6f8ff42dcb70f40b4301540fa221edc318d24b31729901d435e64a9a1"} Jul 28 14:34:57.319598 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:57.319575 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6hz8d" Jul 28 14:34:57.324992 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:34:57.324949 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" podStartSLOduration=1.156501355 podStartE2EDuration="1.324931532s" totalImagesPullingTime="168.430177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:34:57.324370554 +0000 UTC m=+728.125248110" watchObservedRunningTime="2026-07-28 14:34:57.324931532 +0000 UTC m=+728.125809089" Jul 28 14:35:06.492013 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:35:06.491980 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:35:06.492462 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:35:06.492034 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:35:06.504169 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:35:06.504147 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:35:07.380757 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:35:07.380721 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mxppb" Jul 28 14:53:18.917424 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:18.917384 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z"] Jul 28 14:53:18.921176 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:18.921150 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:18.921511 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:18.921478 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52db6ae7-d62d-492e-bd35-44b86bfc5699-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z\" (UID: \"52db6ae7-d62d-492e-bd35-44b86bfc5699\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:18.923829 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:18.923804 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nx5zp\"" Jul 28 14:53:18.927824 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:18.927799 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z"] Jul 28 14:53:19.022741 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:19.022707 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52db6ae7-d62d-492e-bd35-44b86bfc5699-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z\" (UID: \"52db6ae7-d62d-492e-bd35-44b86bfc5699\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:19.023319 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:19.023297 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52db6ae7-d62d-492e-bd35-44b86bfc5699-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z\" (UID: \"52db6ae7-d62d-492e-bd35-44b86bfc5699\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:19.234144 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:19.234064 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:19.377302 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:19.377272 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z"] Jul 28 14:53:19.378960 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:53:19.378934 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52db6ae7_d62d_492e_bd35_44b86bfc5699.slice/crio-c46055981207611595dc17203a5ab96e98c3212ce78d266967bd02e803208ad2 WatchSource:0}: Error finding container c46055981207611595dc17203a5ab96e98c3212ce78d266967bd02e803208ad2: Status 404 returned error can't find the container with id c46055981207611595dc17203a5ab96e98c3212ce78d266967bd02e803208ad2 Jul 28 14:53:19.380897 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:19.380882 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 14:53:20.307807 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:20.307768 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" event={"ID":"52db6ae7-d62d-492e-bd35-44b86bfc5699","Type":"ContainerStarted","Data":"c46055981207611595dc17203a5ab96e98c3212ce78d266967bd02e803208ad2"} Jul 28 14:53:22.322436 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:22.322393 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" event={"ID":"52db6ae7-d62d-492e-bd35-44b86bfc5699","Type":"ContainerStarted","Data":"776de13a2c28e2c030105388d76c2b8a47f6fc03a95cba480b4b4a672f56a8a8"} Jul 28 14:53:22.339747 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:22.339691 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" podStartSLOduration=1.982500371 podStartE2EDuration="4.339673748s" totalImagesPullingTime="2.357173377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:53:22.336967505 +0000 UTC m=+1833.137845060" watchObservedRunningTime="2026-07-28 14:53:22.339673748 +0000 UTC m=+1833.140551304" Jul 28 14:53:23.329193 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:23.329158 2586 generic.go:356] "Generic (PLEG): container finished" podID="52db6ae7-d62d-492e-bd35-44b86bfc5699" containerID="776de13a2c28e2c030105388d76c2b8a47f6fc03a95cba480b4b4a672f56a8a8" exitCode=0 Jul 28 14:53:23.329581 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:23.329250 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" event={"ID":"52db6ae7-d62d-492e-bd35-44b86bfc5699","Type":"ContainerDied","Data":"776de13a2c28e2c030105388d76c2b8a47f6fc03a95cba480b4b4a672f56a8a8"} Jul 28 14:53:24.465894 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:24.465866 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:24.574119 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:24.574068 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52db6ae7-d62d-492e-bd35-44b86bfc5699-maas-api-service-ca\") pod \"52db6ae7-d62d-492e-bd35-44b86bfc5699\" (UID: \"52db6ae7-d62d-492e-bd35-44b86bfc5699\") " Jul 28 14:53:24.574484 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:24.574463 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52db6ae7-d62d-492e-bd35-44b86bfc5699-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52db6ae7-d62d-492e-bd35-44b86bfc5699" (UID: "52db6ae7-d62d-492e-bd35-44b86bfc5699"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:53:24.675755 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:24.675680 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52db6ae7-d62d-492e-bd35-44b86bfc5699-maas-api-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:53:25.341311 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:25.341274 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" event={"ID":"52db6ae7-d62d-492e-bd35-44b86bfc5699","Type":"ContainerDied","Data":"c46055981207611595dc17203a5ab96e98c3212ce78d266967bd02e803208ad2"} Jul 28 14:53:25.341311 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:25.341290 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z" Jul 28 14:53:25.341588 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:25.341322 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c46055981207611595dc17203a5ab96e98c3212ce78d266967bd02e803208ad2" Jul 28 14:53:34.098355 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:34.098320 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z"] Jul 28 14:53:34.100661 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:34.100638 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-856ad4e0-zt57z"] Jul 28 14:53:35.691249 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:53:35.691214 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52db6ae7-d62d-492e-bd35-44b86bfc5699" path="/var/lib/kubelet/pods/52db6ae7-d62d-492e-bd35-44b86bfc5699/volumes" Jul 28 14:54:02.641531 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.641496 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j"] Jul 28 14:54:02.641937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.641924 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52db6ae7-d62d-492e-bd35-44b86bfc5699" containerName="revoke-keys" Jul 28 14:54:02.641937 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.641938 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="52db6ae7-d62d-492e-bd35-44b86bfc5699" containerName="revoke-keys" Jul 28 14:54:02.642031 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.642022 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="52db6ae7-d62d-492e-bd35-44b86bfc5699" containerName="revoke-keys" Jul 28 14:54:02.650119 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.650089 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:02.652445 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.652420 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j"] Jul 28 14:54:02.652652 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.652630 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nx5zp\"" Jul 28 14:54:02.747660 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.747625 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aad66c2f-82ae-47d8-81f9-786b30f565f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j\" (UID: \"aad66c2f-82ae-47d8-81f9-786b30f565f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:02.848413 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.848380 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aad66c2f-82ae-47d8-81f9-786b30f565f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j\" (UID: \"aad66c2f-82ae-47d8-81f9-786b30f565f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:02.849020 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.849000 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aad66c2f-82ae-47d8-81f9-786b30f565f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j\" (UID: \"aad66c2f-82ae-47d8-81f9-786b30f565f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:02.964917 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:02.964835 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:03.096891 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:03.096864 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j"] Jul 28 14:54:03.098262 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:54:03.098236 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaad66c2f_82ae_47d8_81f9_786b30f565f0.slice/crio-7609faa00f5eee635fe4ee4fcbbcd2fe7409e881a4c72f5af901da591e371548 WatchSource:0}: Error finding container 7609faa00f5eee635fe4ee4fcbbcd2fe7409e881a4c72f5af901da591e371548: Status 404 returned error can't find the container with id 7609faa00f5eee635fe4ee4fcbbcd2fe7409e881a4c72f5af901da591e371548 Jul 28 14:54:03.551324 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:03.551278 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" event={"ID":"aad66c2f-82ae-47d8-81f9-786b30f565f0","Type":"ContainerStarted","Data":"e3978d9de087784b61affeb14f2031acea581f33a0422edb91ae3e4f052d0cd9"} Jul 28 14:54:03.551492 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:03.551342 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" event={"ID":"aad66c2f-82ae-47d8-81f9-786b30f565f0","Type":"ContainerStarted","Data":"7609faa00f5eee635fe4ee4fcbbcd2fe7409e881a4c72f5af901da591e371548"} Jul 28 14:54:03.567469 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:03.567414 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" podStartSLOduration=1.567399225 podStartE2EDuration="1.567399225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:54:03.56469448 +0000 UTC m=+1874.365572036" watchObservedRunningTime="2026-07-28 14:54:03.567399225 +0000 UTC m=+1874.368276782" Jul 28 14:54:04.558093 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:04.558059 2586 generic.go:356] "Generic (PLEG): container finished" podID="aad66c2f-82ae-47d8-81f9-786b30f565f0" containerID="e3978d9de087784b61affeb14f2031acea581f33a0422edb91ae3e4f052d0cd9" exitCode=0 Jul 28 14:54:04.558512 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:04.558156 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" event={"ID":"aad66c2f-82ae-47d8-81f9-786b30f565f0","Type":"ContainerDied","Data":"e3978d9de087784b61affeb14f2031acea581f33a0422edb91ae3e4f052d0cd9"} Jul 28 14:54:05.695333 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:05.695303 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:05.776742 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:05.776712 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aad66c2f-82ae-47d8-81f9-786b30f565f0-maas-api-service-ca\") pod \"aad66c2f-82ae-47d8-81f9-786b30f565f0\" (UID: \"aad66c2f-82ae-47d8-81f9-786b30f565f0\") " Jul 28 14:54:05.777099 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:05.777080 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aad66c2f-82ae-47d8-81f9-786b30f565f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aad66c2f-82ae-47d8-81f9-786b30f565f0" (UID: "aad66c2f-82ae-47d8-81f9-786b30f565f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:54:05.877585 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:05.877497 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aad66c2f-82ae-47d8-81f9-786b30f565f0-maas-api-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:54:06.570087 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:06.570055 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" event={"ID":"aad66c2f-82ae-47d8-81f9-786b30f565f0","Type":"ContainerDied","Data":"7609faa00f5eee635fe4ee4fcbbcd2fe7409e881a4c72f5af901da591e371548"} Jul 28 14:54:06.570333 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:06.570102 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7609faa00f5eee635fe4ee4fcbbcd2fe7409e881a4c72f5af901da591e371548" Jul 28 14:54:06.570333 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:06.570065 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j" Jul 28 14:54:09.043052 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:09.043018 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j"] Jul 28 14:54:09.045488 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:09.045464 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4033a9f3-zcz6j"] Jul 28 14:54:09.690962 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:09.690926 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aad66c2f-82ae-47d8-81f9-786b30f565f0" path="/var/lib/kubelet/pods/aad66c2f-82ae-47d8-81f9-786b30f565f0/volumes" Jul 28 14:54:38.707452 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.707372 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm"] Jul 28 14:54:38.707938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.707781 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aad66c2f-82ae-47d8-81f9-786b30f565f0" containerName="revoke-keys" Jul 28 14:54:38.707938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.707792 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="aad66c2f-82ae-47d8-81f9-786b30f565f0" containerName="revoke-keys" Jul 28 14:54:38.707938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.707853 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="aad66c2f-82ae-47d8-81f9-786b30f565f0" containerName="revoke-keys" Jul 28 14:54:38.711030 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.711013 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:38.713302 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.713278 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nx5zp\"" Jul 28 14:54:38.718254 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.718228 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm"] Jul 28 14:54:38.786399 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.786363 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff20c396-27b5-4a50-899a-305442c3ab01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm\" (UID: \"ff20c396-27b5-4a50-899a-305442c3ab01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:38.887946 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.887915 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff20c396-27b5-4a50-899a-305442c3ab01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm\" (UID: \"ff20c396-27b5-4a50-899a-305442c3ab01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:38.888635 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:38.888616 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff20c396-27b5-4a50-899a-305442c3ab01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm\" (UID: \"ff20c396-27b5-4a50-899a-305442c3ab01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:39.022723 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:39.022638 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:39.156657 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:39.156626 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm"] Jul 28 14:54:39.157678 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:54:39.157650 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff20c396_27b5_4a50_899a_305442c3ab01.slice/crio-b1420c7dc9b82a7712af7d6321f0fb7d6f16f7fa20609a42790268ab3b991fe6 WatchSource:0}: Error finding container b1420c7dc9b82a7712af7d6321f0fb7d6f16f7fa20609a42790268ab3b991fe6: Status 404 returned error can't find the container with id b1420c7dc9b82a7712af7d6321f0fb7d6f16f7fa20609a42790268ab3b991fe6 Jul 28 14:54:39.755219 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:39.755144 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" event={"ID":"ff20c396-27b5-4a50-899a-305442c3ab01","Type":"ContainerStarted","Data":"eab3b7c845546e63406e21c9736f5ca02e4bac7d029674addc246fb666a44950"} Jul 28 14:54:39.755219 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:39.755197 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" event={"ID":"ff20c396-27b5-4a50-899a-305442c3ab01","Type":"ContainerStarted","Data":"b1420c7dc9b82a7712af7d6321f0fb7d6f16f7fa20609a42790268ab3b991fe6"} Jul 28 14:54:39.770147 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:39.770099 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" podStartSLOduration=1.770083278 podStartE2EDuration="1.770083278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:54:39.768604373 +0000 UTC m=+1910.569481929" watchObservedRunningTime="2026-07-28 14:54:39.770083278 +0000 UTC m=+1910.570960833" Jul 28 14:54:40.761894 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:40.761861 2586 generic.go:356] "Generic (PLEG): container finished" podID="ff20c396-27b5-4a50-899a-305442c3ab01" containerID="eab3b7c845546e63406e21c9736f5ca02e4bac7d029674addc246fb666a44950" exitCode=0 Jul 28 14:54:40.762304 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:40.761947 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" event={"ID":"ff20c396-27b5-4a50-899a-305442c3ab01","Type":"ContainerDied","Data":"eab3b7c845546e63406e21c9736f5ca02e4bac7d029674addc246fb666a44950"} Jul 28 14:54:41.895909 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:41.895887 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:41.919226 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:41.919202 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff20c396-27b5-4a50-899a-305442c3ab01-maas-api-service-ca\") pod \"ff20c396-27b5-4a50-899a-305442c3ab01\" (UID: \"ff20c396-27b5-4a50-899a-305442c3ab01\") " Jul 28 14:54:41.919602 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:41.919572 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff20c396-27b5-4a50-899a-305442c3ab01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff20c396-27b5-4a50-899a-305442c3ab01" (UID: "ff20c396-27b5-4a50-899a-305442c3ab01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:54:42.020295 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:42.020221 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff20c396-27b5-4a50-899a-305442c3ab01-maas-api-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:54:42.773970 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:42.773940 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" Jul 28 14:54:42.773970 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:42.773948 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm" event={"ID":"ff20c396-27b5-4a50-899a-305442c3ab01","Type":"ContainerDied","Data":"b1420c7dc9b82a7712af7d6321f0fb7d6f16f7fa20609a42790268ab3b991fe6"} Jul 28 14:54:42.774179 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:42.773998 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1420c7dc9b82a7712af7d6321f0fb7d6f16f7fa20609a42790268ab3b991fe6" Jul 28 14:54:43.663186 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:43.663150 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm"] Jul 28 14:54:43.667672 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:43.667648 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-38ffd343-l2tnm"] Jul 28 14:54:43.693375 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:54:43.693343 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff20c396-27b5-4a50-899a-305442c3ab01" path="/var/lib/kubelet/pods/ff20c396-27b5-4a50-899a-305442c3ab01/volumes" Jul 28 14:55:12.797405 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.797369 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7"] Jul 28 14:55:12.797895 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.797786 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff20c396-27b5-4a50-899a-305442c3ab01" containerName="revoke-keys" Jul 28 14:55:12.797895 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.797798 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff20c396-27b5-4a50-899a-305442c3ab01" containerName="revoke-keys" Jul 28 14:55:12.797895 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.797873 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff20c396-27b5-4a50-899a-305442c3ab01" containerName="revoke-keys" Jul 28 14:55:12.801112 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.801097 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:12.803606 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.803586 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nx5zp\"" Jul 28 14:55:12.810442 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.810420 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7"] Jul 28 14:55:12.904331 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:12.904302 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307e2bf0-70be-4251-82ea-ced7d739e5c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7\" (UID: \"307e2bf0-70be-4251-82ea-ced7d739e5c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:13.005650 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.005621 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307e2bf0-70be-4251-82ea-ced7d739e5c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7\" (UID: \"307e2bf0-70be-4251-82ea-ced7d739e5c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:13.006272 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.006252 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307e2bf0-70be-4251-82ea-ced7d739e5c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7\" (UID: \"307e2bf0-70be-4251-82ea-ced7d739e5c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:13.112643 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.112614 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:13.260290 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.260262 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7"] Jul 28 14:55:13.262168 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:55:13.262134 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307e2bf0_70be_4251_82ea_ced7d739e5c8.slice/crio-745c2d6da0b446acff598ae1e781be33dac9556c0dd2759b1707399bc33cca86 WatchSource:0}: Error finding container 745c2d6da0b446acff598ae1e781be33dac9556c0dd2759b1707399bc33cca86: Status 404 returned error can't find the container with id 745c2d6da0b446acff598ae1e781be33dac9556c0dd2759b1707399bc33cca86 Jul 28 14:55:13.956965 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.956931 2586 generic.go:356] "Generic (PLEG): container finished" podID="307e2bf0-70be-4251-82ea-ced7d739e5c8" containerID="e82a34bcc385468d03d33319fbd4b21b2972798b4e84a8abef346c987b7d020a" exitCode=0 Jul 28 14:55:13.957326 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.957010 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" event={"ID":"307e2bf0-70be-4251-82ea-ced7d739e5c8","Type":"ContainerDied","Data":"e82a34bcc385468d03d33319fbd4b21b2972798b4e84a8abef346c987b7d020a"} Jul 28 14:55:13.957326 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:13.957078 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" event={"ID":"307e2bf0-70be-4251-82ea-ced7d739e5c8","Type":"ContainerStarted","Data":"745c2d6da0b446acff598ae1e781be33dac9556c0dd2759b1707399bc33cca86"} Jul 28 14:55:15.116698 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.116670 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:15.227824 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.227739 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307e2bf0-70be-4251-82ea-ced7d739e5c8-maas-api-service-ca\") pod \"307e2bf0-70be-4251-82ea-ced7d739e5c8\" (UID: \"307e2bf0-70be-4251-82ea-ced7d739e5c8\") " Jul 28 14:55:15.228087 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.228065 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307e2bf0-70be-4251-82ea-ced7d739e5c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "307e2bf0-70be-4251-82ea-ced7d739e5c8" (UID: "307e2bf0-70be-4251-82ea-ced7d739e5c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:55:15.328799 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.328763 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/307e2bf0-70be-4251-82ea-ced7d739e5c8-maas-api-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:55:15.969732 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.969647 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" Jul 28 14:55:15.969732 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.969672 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7" event={"ID":"307e2bf0-70be-4251-82ea-ced7d739e5c8","Type":"ContainerDied","Data":"745c2d6da0b446acff598ae1e781be33dac9556c0dd2759b1707399bc33cca86"} Jul 28 14:55:15.969964 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:15.969746 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="745c2d6da0b446acff598ae1e781be33dac9556c0dd2759b1707399bc33cca86" Jul 28 14:55:18.817672 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:18.817632 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7"] Jul 28 14:55:18.820087 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:18.820061 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a0bd8cb8-bf9w7"] Jul 28 14:55:19.692812 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:55:19.692779 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="307e2bf0-70be-4251-82ea-ced7d739e5c8" path="/var/lib/kubelet/pods/307e2bf0-70be-4251-82ea-ced7d739e5c8/volumes" Jul 28 14:57:37.770500 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.770468 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff"] Jul 28 14:57:37.770938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.770851 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="307e2bf0-70be-4251-82ea-ced7d739e5c8" containerName="revoke-keys" Jul 28 14:57:37.770938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.770862 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="307e2bf0-70be-4251-82ea-ced7d739e5c8" containerName="revoke-keys" Jul 28 14:57:37.770938 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.770931 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="307e2bf0-70be-4251-82ea-ced7d739e5c8" containerName="revoke-keys" Jul 28 14:57:37.773704 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.773689 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:37.776535 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.776513 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nx5zp\"" Jul 28 14:57:37.780318 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.780296 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff"] Jul 28 14:57:37.842340 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.842312 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd66b063-852d-4fcc-8cde-20ef8db9d1a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-efb491e3-skmff\" (UID: \"dd66b063-852d-4fcc-8cde-20ef8db9d1a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:37.943211 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.943188 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd66b063-852d-4fcc-8cde-20ef8db9d1a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-efb491e3-skmff\" (UID: \"dd66b063-852d-4fcc-8cde-20ef8db9d1a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:37.943789 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:37.943766 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd66b063-852d-4fcc-8cde-20ef8db9d1a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-efb491e3-skmff\" (UID: \"dd66b063-852d-4fcc-8cde-20ef8db9d1a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:38.086348 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:38.086325 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:38.421161 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:38.421137 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff"] Jul 28 14:57:38.422126 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:57:38.422098 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd66b063_852d_4fcc_8cde_20ef8db9d1a3.slice/crio-c3572553cc56f90920b9a6f9d84131c489ebc0f69f63f5e427183127cd81d62c WatchSource:0}: Error finding container c3572553cc56f90920b9a6f9d84131c489ebc0f69f63f5e427183127cd81d62c: Status 404 returned error can't find the container with id c3572553cc56f90920b9a6f9d84131c489ebc0f69f63f5e427183127cd81d62c Jul 28 14:57:38.764855 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:38.764822 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" event={"ID":"dd66b063-852d-4fcc-8cde-20ef8db9d1a3","Type":"ContainerStarted","Data":"3d54e308dd5d1b7545fbfe0fcae1cbfe83d47f2bde86d3bbe23968bcc63f8fd6"} Jul 28 14:57:38.764980 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:38.764876 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" event={"ID":"dd66b063-852d-4fcc-8cde-20ef8db9d1a3","Type":"ContainerStarted","Data":"c3572553cc56f90920b9a6f9d84131c489ebc0f69f63f5e427183127cd81d62c"} Jul 28 14:57:38.782618 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:38.782578 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" podStartSLOduration=1.782565339 podStartE2EDuration="1.782565339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:57:38.780279948 +0000 UTC m=+2089.581157503" watchObservedRunningTime="2026-07-28 14:57:38.782565339 +0000 UTC m=+2089.583442890" Jul 28 14:57:39.772269 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:39.772239 2586 generic.go:356] "Generic (PLEG): container finished" podID="dd66b063-852d-4fcc-8cde-20ef8db9d1a3" containerID="3d54e308dd5d1b7545fbfe0fcae1cbfe83d47f2bde86d3bbe23968bcc63f8fd6" exitCode=0 Jul 28 14:57:39.772434 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:39.772302 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" event={"ID":"dd66b063-852d-4fcc-8cde-20ef8db9d1a3","Type":"ContainerDied","Data":"3d54e308dd5d1b7545fbfe0fcae1cbfe83d47f2bde86d3bbe23968bcc63f8fd6"} Jul 28 14:57:40.907143 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:40.907116 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:40.965584 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:40.965545 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd66b063-852d-4fcc-8cde-20ef8db9d1a3-maas-api-service-ca\") pod \"dd66b063-852d-4fcc-8cde-20ef8db9d1a3\" (UID: \"dd66b063-852d-4fcc-8cde-20ef8db9d1a3\") " Jul 28 14:57:40.965848 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:40.965827 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd66b063-852d-4fcc-8cde-20ef8db9d1a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd66b063-852d-4fcc-8cde-20ef8db9d1a3" (UID: "dd66b063-852d-4fcc-8cde-20ef8db9d1a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:57:41.067066 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:41.067046 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd66b063-852d-4fcc-8cde-20ef8db9d1a3-maas-api-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:57:41.784025 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:41.784000 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" Jul 28 14:57:41.784173 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:41.784018 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff" event={"ID":"dd66b063-852d-4fcc-8cde-20ef8db9d1a3","Type":"ContainerDied","Data":"c3572553cc56f90920b9a6f9d84131c489ebc0f69f63f5e427183127cd81d62c"} Jul 28 14:57:41.784173 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:41.784059 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3572553cc56f90920b9a6f9d84131c489ebc0f69f63f5e427183127cd81d62c" Jul 28 14:57:48.314433 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:48.314400 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff"] Jul 28 14:57:48.316730 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:48.316525 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-efb491e3-skmff"] Jul 28 14:57:49.697304 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:57:49.697271 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd66b063-852d-4fcc-8cde-20ef8db9d1a3" path="/var/lib/kubelet/pods/dd66b063-852d-4fcc-8cde-20ef8db9d1a3/volumes" Jul 28 14:59:13.236046 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.236012 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz"] Jul 28 14:59:13.236441 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.236395 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd66b063-852d-4fcc-8cde-20ef8db9d1a3" containerName="revoke-keys" Jul 28 14:59:13.236441 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.236406 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd66b063-852d-4fcc-8cde-20ef8db9d1a3" containerName="revoke-keys" Jul 28 14:59:13.236514 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.236463 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd66b063-852d-4fcc-8cde-20ef8db9d1a3" containerName="revoke-keys" Jul 28 14:59:13.239473 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.239458 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:13.241785 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.241766 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nx5zp\"" Jul 28 14:59:13.247500 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.246880 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz"] Jul 28 14:59:13.352462 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.352433 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467d8481-7801-4c23-a14f-93d593b11f00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-96a65772-zglvz\" (UID: \"467d8481-7801-4c23-a14f-93d593b11f00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:13.453693 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.453658 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467d8481-7801-4c23-a14f-93d593b11f00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-96a65772-zglvz\" (UID: \"467d8481-7801-4c23-a14f-93d593b11f00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:13.454306 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.454281 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467d8481-7801-4c23-a14f-93d593b11f00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-96a65772-zglvz\" (UID: \"467d8481-7801-4c23-a14f-93d593b11f00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:13.551499 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.551431 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:13.681311 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:13.681268 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz"] Jul 28 14:59:13.683210 ip-10-0-131-88 kubenswrapper[2586]: W0728 14:59:13.683187 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467d8481_7801_4c23_a14f_93d593b11f00.slice/crio-879e28eef4dd61689fdbce9feea24a589e6795ca25ee5354081188248fde3707 WatchSource:0}: Error finding container 879e28eef4dd61689fdbce9feea24a589e6795ca25ee5354081188248fde3707: Status 404 returned error can't find the container with id 879e28eef4dd61689fdbce9feea24a589e6795ca25ee5354081188248fde3707 Jul 28 14:59:14.305415 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:14.305387 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" event={"ID":"467d8481-7801-4c23-a14f-93d593b11f00","Type":"ContainerStarted","Data":"c4f335182cfc96c3781b04f864942c28adcb29cf0cd851ba82843c255eb9d5ce"} Jul 28 14:59:14.305799 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:14.305437 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" event={"ID":"467d8481-7801-4c23-a14f-93d593b11f00","Type":"ContainerStarted","Data":"879e28eef4dd61689fdbce9feea24a589e6795ca25ee5354081188248fde3707"} Jul 28 14:59:14.319215 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:14.319177 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" podStartSLOduration=1.319163379 podStartE2EDuration="1.319163379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 14:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 14:59:14.318076457 +0000 UTC m=+2185.118954013" watchObservedRunningTime="2026-07-28 14:59:14.319163379 +0000 UTC m=+2185.120040934" Jul 28 14:59:15.312345 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:15.312310 2586 generic.go:356] "Generic (PLEG): container finished" podID="467d8481-7801-4c23-a14f-93d593b11f00" containerID="c4f335182cfc96c3781b04f864942c28adcb29cf0cd851ba82843c255eb9d5ce" exitCode=0 Jul 28 14:59:15.312784 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:15.312373 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" event={"ID":"467d8481-7801-4c23-a14f-93d593b11f00","Type":"ContainerDied","Data":"c4f335182cfc96c3781b04f864942c28adcb29cf0cd851ba82843c255eb9d5ce"} Jul 28 14:59:16.443634 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:16.443611 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:16.582853 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:16.582776 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467d8481-7801-4c23-a14f-93d593b11f00-maas-api-service-ca\") pod \"467d8481-7801-4c23-a14f-93d593b11f00\" (UID: \"467d8481-7801-4c23-a14f-93d593b11f00\") " Jul 28 14:59:16.583120 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:16.583097 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467d8481-7801-4c23-a14f-93d593b11f00-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "467d8481-7801-4c23-a14f-93d593b11f00" (UID: "467d8481-7801-4c23-a14f-93d593b11f00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 14:59:16.684373 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:16.684343 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467d8481-7801-4c23-a14f-93d593b11f00-maas-api-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 14:59:17.325498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:17.325469 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" Jul 28 14:59:17.325498 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:17.325481 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz" event={"ID":"467d8481-7801-4c23-a14f-93d593b11f00","Type":"ContainerDied","Data":"879e28eef4dd61689fdbce9feea24a589e6795ca25ee5354081188248fde3707"} Jul 28 14:59:17.325722 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:17.325531 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="879e28eef4dd61689fdbce9feea24a589e6795ca25ee5354081188248fde3707" Jul 28 14:59:29.048030 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:29.047994 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz"] Jul 28 14:59:29.050193 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:29.050170 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-96a65772-zglvz"] Jul 28 14:59:29.691540 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:29.691505 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467d8481-7801-4c23-a14f-93d593b11f00" path="/var/lib/kubelet/pods/467d8481-7801-4c23-a14f-93d593b11f00/volumes" Jul 28 14:59:49.883017 ip-10-0-131-88 kubenswrapper[2586]: I0728 14:59:49.882980 2586 scope.go:117] "RemoveContainer" containerID="776de13a2c28e2c030105388d76c2b8a47f6fc03a95cba480b4b4a672f56a8a8" Jul 28 15:00:49.904468 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:00:49.904439 2586 scope.go:117] "RemoveContainer" containerID="e3978d9de087784b61affeb14f2031acea581f33a0422edb91ae3e4f052d0cd9" Jul 28 15:00:49.912894 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:00:49.912873 2586 scope.go:117] "RemoveContainer" containerID="eab3b7c845546e63406e21c9736f5ca02e4bac7d029674addc246fb666a44950" Jul 28 15:01:49.936392 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:01:49.936363 2586 scope.go:117] "RemoveContainer" containerID="e82a34bcc385468d03d33319fbd4b21b2972798b4e84a8abef346c987b7d020a" Jul 28 15:02:15.869020 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.868979 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf"] Jul 28 15:02:15.869935 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.869910 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467d8481-7801-4c23-a14f-93d593b11f00" containerName="revoke-keys" Jul 28 15:02:15.869935 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.869937 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="467d8481-7801-4c23-a14f-93d593b11f00" containerName="revoke-keys" Jul 28 15:02:15.870143 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.870130 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="467d8481-7801-4c23-a14f-93d593b11f00" containerName="revoke-keys" Jul 28 15:02:15.874320 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.874298 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:15.877965 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.877938 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-904baa-serving-cert\"" Jul 28 15:02:15.878080 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.877975 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bk5gs\"" Jul 28 15:02:15.880758 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.880735 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf"] Jul 28 15:02:15.918750 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.918723 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:15.918843 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.918760 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:15.918843 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:15.918778 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4gnk\" (UniqueName: \"kubernetes.io/projected/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-kube-api-access-t4gnk\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.019697 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.019670 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.019837 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.019709 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.019837 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.019730 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4gnk\" (UniqueName: \"kubernetes.io/projected/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-kube-api-access-t4gnk\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.019914 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:02:16.019830 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-serving-cert: secret "maas-api-e2e-inf-b-904baa-serving-cert" not found Jul 28 15:02:16.019914 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:02:16.019911 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls podName:f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3 nodeName:}" failed. No retries permitted until 2026-07-28 15:02:16.519889064 +0000 UTC m=+2367.320766598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls") pod "maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" (UID: "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3") : secret "maas-api-e2e-inf-b-904baa-serving-cert" not found Jul 28 15:02:16.020365 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.020347 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.044081 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.044056 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4gnk\" (UniqueName: \"kubernetes.io/projected/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-kube-api-access-t4gnk\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.523366 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.523337 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.525910 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.525883 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls\") pod \"maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.787011 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.786932 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:16.918342 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.918310 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf"] Jul 28 15:02:16.919674 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:02:16.919648 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a2b38e_b3c5_42ff_9521_53db9b3b5ed3.slice/crio-00fd05e6dffb5863860feca4e6489150d3a6acbd9065f758c02ce44332e2b6ca WatchSource:0}: Error finding container 00fd05e6dffb5863860feca4e6489150d3a6acbd9065f758c02ce44332e2b6ca: Status 404 returned error can't find the container with id 00fd05e6dffb5863860feca4e6489150d3a6acbd9065f758c02ce44332e2b6ca Jul 28 15:02:16.921123 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:16.921101 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 15:02:17.336207 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:17.336172 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" event={"ID":"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3","Type":"ContainerStarted","Data":"00fd05e6dffb5863860feca4e6489150d3a6acbd9065f758c02ce44332e2b6ca"} Jul 28 15:02:18.342611 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:18.342573 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" event={"ID":"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3","Type":"ContainerStarted","Data":"ad890c60db5fe49c6655e9e6c634ef8d5c7d4fdd3792e2226a75ae2455afcf10"} Jul 28 15:02:18.342984 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:18.342696 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:02:18.358735 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:18.358683 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" podStartSLOduration=2.7964532909999997 podStartE2EDuration="3.358667803s" totalImagesPullingTime="562.214512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:02:18.357977255 +0000 UTC m=+2369.158854811" watchObservedRunningTime="2026-07-28 15:02:18.358667803 +0000 UTC m=+2369.159545360" Jul 28 15:02:24.353487 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:02:24.353460 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:03:49.967809 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:03:49.967775 2586 scope.go:117] "RemoveContainer" containerID="3d54e308dd5d1b7545fbfe0fcae1cbfe83d47f2bde86d3bbe23968bcc63f8fd6" Jul 28 15:04:19.634336 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:19.634292 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf"] Jul 28 15:04:19.634836 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:19.634563 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" podUID="f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" containerName="maas-api" containerID="cri-o://ad890c60db5fe49c6655e9e6c634ef8d5c7d4fdd3792e2226a75ae2455afcf10" gracePeriod=30 Jul 28 15:04:20.040353 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.040319 2586 generic.go:356] "Generic (PLEG): container finished" podID="f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" containerID="ad890c60db5fe49c6655e9e6c634ef8d5c7d4fdd3792e2226a75ae2455afcf10" exitCode=0 Jul 28 15:04:20.040540 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.040389 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" event={"ID":"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3","Type":"ContainerDied","Data":"ad890c60db5fe49c6655e9e6c634ef8d5c7d4fdd3792e2226a75ae2455afcf10"} Jul 28 15:04:20.096813 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.096788 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:04:20.190830 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.190685 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls\") pod \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " Jul 28 15:04:20.190830 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.190740 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4gnk\" (UniqueName: \"kubernetes.io/projected/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-kube-api-access-t4gnk\") pod \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " Jul 28 15:04:20.190830 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.190783 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-openshift-service-ca\") pod \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\" (UID: \"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3\") " Jul 28 15:04:20.191174 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.191150 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" (UID: "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:04:20.192951 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.192930 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" (UID: "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 15:04:20.193030 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.192929 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-kube-api-access-t4gnk" (OuterVolumeSpecName: "kube-api-access-t4gnk") pod "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" (UID: "f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3"). InnerVolumeSpecName "kube-api-access-t4gnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:04:20.292437 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.292407 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-maas-api-tls\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:04:20.292437 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.292433 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4gnk\" (UniqueName: \"kubernetes.io/projected/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-kube-api-access-t4gnk\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:04:20.292437 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:20.292443 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3-openshift-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:04:21.047223 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:21.047190 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" Jul 28 15:04:21.047813 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:21.047223 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf" event={"ID":"f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3","Type":"ContainerDied","Data":"00fd05e6dffb5863860feca4e6489150d3a6acbd9065f758c02ce44332e2b6ca"} Jul 28 15:04:21.047813 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:21.047299 2586 scope.go:117] "RemoveContainer" containerID="ad890c60db5fe49c6655e9e6c634ef8d5c7d4fdd3792e2226a75ae2455afcf10" Jul 28 15:04:21.067954 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:21.067928 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf"] Jul 28 15:04:21.070193 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:21.070175 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-904baa-68cf5dd584-wv2gf"] Jul 28 15:04:21.691460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:21.691420 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" path="/var/lib/kubelet/pods/f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3/volumes" Jul 28 15:04:59.610061 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.610030 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8"] Jul 28 15:04:59.610535 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.610419 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" containerName="maas-api" Jul 28 15:04:59.610535 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.610431 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" containerName="maas-api" Jul 28 15:04:59.610535 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.610521 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a2b38e-b3c5-42ff-9521-53db9b3b5ed3" containerName="maas-api" Jul 28 15:04:59.623523 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.623490 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.624773 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.624750 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8"] Jul 28 15:04:59.626257 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.626234 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-84fa8d-serving-cert\"" Jul 28 15:04:59.626380 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.626245 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bk5gs\"" Jul 28 15:04:59.770987 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.770946 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w5sm\" (UniqueName: \"kubernetes.io/projected/faeee89f-d121-446a-9a7e-c27fa4eba183-kube-api-access-8w5sm\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.771190 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.771127 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.771269 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.771193 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faeee89f-d121-446a-9a7e-c27fa4eba183-openshift-service-ca\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.872343 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.872241 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w5sm\" (UniqueName: \"kubernetes.io/projected/faeee89f-d121-446a-9a7e-c27fa4eba183-kube-api-access-8w5sm\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.872343 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.872310 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.872585 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.872349 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faeee89f-d121-446a-9a7e-c27fa4eba183-openshift-service-ca\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.872585 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:04:59.872451 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-serving-cert: secret "maas-api-e2e-api-b-84fa8d-serving-cert" not found Jul 28 15:04:59.872585 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:04:59.872515 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls podName:faeee89f-d121-446a-9a7e-c27fa4eba183 nodeName:}" failed. No retries permitted until 2026-07-28 15:05:00.372498049 +0000 UTC m=+2531.173375583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls") pod "maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" (UID: "faeee89f-d121-446a-9a7e-c27fa4eba183") : secret "maas-api-e2e-api-b-84fa8d-serving-cert" not found Jul 28 15:04:59.873064 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.873047 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faeee89f-d121-446a-9a7e-c27fa4eba183-openshift-service-ca\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:04:59.883310 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:04:59.883281 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w5sm\" (UniqueName: \"kubernetes.io/projected/faeee89f-d121-446a-9a7e-c27fa4eba183-kube-api-access-8w5sm\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:05:00.377609 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.377571 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:05:00.380140 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.380104 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls\") pod \"maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:05:00.388683 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.388661 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj"] Jul 28 15:05:00.392334 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.392320 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.394753 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.394734 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-84fa8d\"" Jul 28 15:05:00.394855 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.394736 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-84fa8d-dockercfg-xdksq\"" Jul 28 15:05:00.399659 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.399639 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj"] Jul 28 15:05:00.478225 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.478186 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27807ae5-c540-4a08-93a6-e6948a3c7510-plugins-config-volume\") pod \"payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.478423 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.478248 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tkld\" (UniqueName: \"kubernetes.io/projected/27807ae5-c540-4a08-93a6-e6948a3c7510-kube-api-access-4tkld\") pod \"payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.535401 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.535366 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:05:00.579900 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.579865 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27807ae5-c540-4a08-93a6-e6948a3c7510-plugins-config-volume\") pod \"payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.579900 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.579907 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tkld\" (UniqueName: \"kubernetes.io/projected/27807ae5-c540-4a08-93a6-e6948a3c7510-kube-api-access-4tkld\") pod \"payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.580588 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.580546 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27807ae5-c540-4a08-93a6-e6948a3c7510-plugins-config-volume\") pod \"payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.588486 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.588458 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tkld\" (UniqueName: \"kubernetes.io/projected/27807ae5-c540-4a08-93a6-e6948a3c7510-kube-api-access-4tkld\") pod \"payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.674052 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.674023 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8"] Jul 28 15:05:00.675215 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:05:00.675177 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaeee89f_d121_446a_9a7e_c27fa4eba183.slice/crio-8b973c733fe13186dd412eeb47aa493a733bd598eaee69183f1a4c1d7429e584 WatchSource:0}: Error finding container 8b973c733fe13186dd412eeb47aa493a733bd598eaee69183f1a4c1d7429e584: Status 404 returned error can't find the container with id 8b973c733fe13186dd412eeb47aa493a733bd598eaee69183f1a4c1d7429e584 Jul 28 15:05:00.705643 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.705606 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:00.842582 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:00.842536 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj"] Jul 28 15:05:00.844004 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:05:00.843976 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27807ae5_c540_4a08_93a6_e6948a3c7510.slice/crio-4c603cfaf9c302f5b5590dcd92ebca3e2de186ae69bbf333fda77dc4070d6a8a WatchSource:0}: Error finding container 4c603cfaf9c302f5b5590dcd92ebca3e2de186ae69bbf333fda77dc4070d6a8a: Status 404 returned error can't find the container with id 4c603cfaf9c302f5b5590dcd92ebca3e2de186ae69bbf333fda77dc4070d6a8a Jul 28 15:05:01.279378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.279274 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" event={"ID":"faeee89f-d121-446a-9a7e-c27fa4eba183","Type":"ContainerStarted","Data":"73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b"} Jul 28 15:05:01.279378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.279336 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" event={"ID":"faeee89f-d121-446a-9a7e-c27fa4eba183","Type":"ContainerStarted","Data":"8b973c733fe13186dd412eeb47aa493a733bd598eaee69183f1a4c1d7429e584"} Jul 28 15:05:01.279378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.279362 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:05:01.280945 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.280915 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" event={"ID":"27807ae5-c540-4a08-93a6-e6948a3c7510","Type":"ContainerStarted","Data":"72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611"} Jul 28 15:05:01.281076 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.280963 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" event={"ID":"27807ae5-c540-4a08-93a6-e6948a3c7510","Type":"ContainerStarted","Data":"4c603cfaf9c302f5b5590dcd92ebca3e2de186ae69bbf333fda77dc4070d6a8a"} Jul 28 15:05:01.281076 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.281002 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:01.296169 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.296123 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" podStartSLOduration=1.9422660729999999 podStartE2EDuration="2.296109511s" totalImagesPullingTime="353.843438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:05:01.294358729 +0000 UTC m=+2532.095236285" watchObservedRunningTime="2026-07-28 15:05:01.296109511 +0000 UTC m=+2532.096987066" Jul 28 15:05:01.310962 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:01.310906 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" podStartSLOduration=1.31089152 podStartE2EDuration="1.31089152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:05:01.308984438 +0000 UTC m=+2532.109861994" watchObservedRunningTime="2026-07-28 15:05:01.31089152 +0000 UTC m=+2532.111769094" Jul 28 15:05:07.292476 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:07.292447 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:05:12.289199 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:12.289168 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:05:50.004454 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:05:50.004414 2586 scope.go:117] "RemoveContainer" containerID="c4f335182cfc96c3781b04f864942c28adcb29cf0cd851ba82843c255eb9d5ce" Jul 28 15:06:06.558631 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:06.558592 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8"] Jul 28 15:06:06.559202 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:06.558897 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" podUID="faeee89f-d121-446a-9a7e-c27fa4eba183" containerName="maas-api" containerID="cri-o://73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b" gracePeriod=30 Jul 28 15:06:06.607750 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:06.607715 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj"] Jul 28 15:06:06.608032 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:06.607975 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" podUID="27807ae5-c540-4a08-93a6-e6948a3c7510" containerName="payload-processing" containerID="cri-o://72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611" gracePeriod=30 Jul 28 15:06:06.869770 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:06.869747 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:06:06.892392 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:06.892368 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:06:07.012977 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.012922 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w5sm\" (UniqueName: \"kubernetes.io/projected/faeee89f-d121-446a-9a7e-c27fa4eba183-kube-api-access-8w5sm\") pod \"faeee89f-d121-446a-9a7e-c27fa4eba183\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " Jul 28 15:06:07.012977 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.012988 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls\") pod \"faeee89f-d121-446a-9a7e-c27fa4eba183\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " Jul 28 15:06:07.013240 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.013049 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faeee89f-d121-446a-9a7e-c27fa4eba183-openshift-service-ca\") pod \"faeee89f-d121-446a-9a7e-c27fa4eba183\" (UID: \"faeee89f-d121-446a-9a7e-c27fa4eba183\") " Jul 28 15:06:07.013240 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.013080 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tkld\" (UniqueName: \"kubernetes.io/projected/27807ae5-c540-4a08-93a6-e6948a3c7510-kube-api-access-4tkld\") pod \"27807ae5-c540-4a08-93a6-e6948a3c7510\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " Jul 28 15:06:07.013240 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.013133 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27807ae5-c540-4a08-93a6-e6948a3c7510-plugins-config-volume\") pod \"27807ae5-c540-4a08-93a6-e6948a3c7510\" (UID: \"27807ae5-c540-4a08-93a6-e6948a3c7510\") " Jul 28 15:06:07.013467 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.013430 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faeee89f-d121-446a-9a7e-c27fa4eba183-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "faeee89f-d121-446a-9a7e-c27fa4eba183" (UID: "faeee89f-d121-446a-9a7e-c27fa4eba183"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:06:07.013634 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.013610 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27807ae5-c540-4a08-93a6-e6948a3c7510-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "27807ae5-c540-4a08-93a6-e6948a3c7510" (UID: "27807ae5-c540-4a08-93a6-e6948a3c7510"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:06:07.015278 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.015254 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "faeee89f-d121-446a-9a7e-c27fa4eba183" (UID: "faeee89f-d121-446a-9a7e-c27fa4eba183"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 15:06:07.015363 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.015291 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faeee89f-d121-446a-9a7e-c27fa4eba183-kube-api-access-8w5sm" (OuterVolumeSpecName: "kube-api-access-8w5sm") pod "faeee89f-d121-446a-9a7e-c27fa4eba183" (UID: "faeee89f-d121-446a-9a7e-c27fa4eba183"). InnerVolumeSpecName "kube-api-access-8w5sm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:06:07.015363 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.015348 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27807ae5-c540-4a08-93a6-e6948a3c7510-kube-api-access-4tkld" (OuterVolumeSpecName: "kube-api-access-4tkld") pod "27807ae5-c540-4a08-93a6-e6948a3c7510" (UID: "27807ae5-c540-4a08-93a6-e6948a3c7510"). InnerVolumeSpecName "kube-api-access-4tkld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:06:07.114243 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.114143 2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27807ae5-c540-4a08-93a6-e6948a3c7510-plugins-config-volume\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:06:07.114243 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.114176 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w5sm\" (UniqueName: \"kubernetes.io/projected/faeee89f-d121-446a-9a7e-c27fa4eba183-kube-api-access-8w5sm\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:06:07.114243 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.114187 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faeee89f-d121-446a-9a7e-c27fa4eba183-maas-api-tls\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:06:07.114243 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.114197 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faeee89f-d121-446a-9a7e-c27fa4eba183-openshift-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:06:07.114243 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.114206 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tkld\" (UniqueName: \"kubernetes.io/projected/27807ae5-c540-4a08-93a6-e6948a3c7510-kube-api-access-4tkld\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:06:07.656234 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.656196 2586 generic.go:356] "Generic (PLEG): container finished" podID="27807ae5-c540-4a08-93a6-e6948a3c7510" containerID="72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611" exitCode=0 Jul 28 15:06:07.656952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.656241 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" Jul 28 15:06:07.656952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.656282 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" event={"ID":"27807ae5-c540-4a08-93a6-e6948a3c7510","Type":"ContainerDied","Data":"72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611"} Jul 28 15:06:07.656952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.656338 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj" event={"ID":"27807ae5-c540-4a08-93a6-e6948a3c7510","Type":"ContainerDied","Data":"4c603cfaf9c302f5b5590dcd92ebca3e2de186ae69bbf333fda77dc4070d6a8a"} Jul 28 15:06:07.656952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.656361 2586 scope.go:117] "RemoveContainer" containerID="72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611" Jul 28 15:06:07.657761 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.657732 2586 generic.go:356] "Generic (PLEG): container finished" podID="faeee89f-d121-446a-9a7e-c27fa4eba183" containerID="73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b" exitCode=0 Jul 28 15:06:07.657818 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.657786 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" event={"ID":"faeee89f-d121-446a-9a7e-c27fa4eba183","Type":"ContainerDied","Data":"73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b"} Jul 28 15:06:07.657818 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.657795 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" Jul 28 15:06:07.657889 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.657821 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8" event={"ID":"faeee89f-d121-446a-9a7e-c27fa4eba183","Type":"ContainerDied","Data":"8b973c733fe13186dd412eeb47aa493a733bd598eaee69183f1a4c1d7429e584"} Jul 28 15:06:07.669758 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.669483 2586 scope.go:117] "RemoveContainer" containerID="72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611" Jul 28 15:06:07.670601 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:06:07.670537 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611\": container with ID starting with 72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611 not found: ID does not exist" containerID="72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611" Jul 28 15:06:07.670723 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.670614 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611"} err="failed to get container status \"72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611\": rpc error: code = NotFound desc = could not find container \"72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611\": container with ID starting with 72d8348b99e682adb13de5b3e717571908a865bafc6139b1be564324f60ca611 not found: ID does not exist" Jul 28 15:06:07.670723 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.670638 2586 scope.go:117] "RemoveContainer" containerID="73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b" Jul 28 15:06:07.677415 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.677394 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj"] Jul 28 15:06:07.680647 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.680628 2586 scope.go:117] "RemoveContainer" containerID="73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b" Jul 28 15:06:07.680743 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.680728 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84fa8d-8cc967f6-g7wgj"] Jul 28 15:06:07.680945 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:06:07.680921 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b\": container with ID starting with 73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b not found: ID does not exist" containerID="73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b" Jul 28 15:06:07.680990 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.680950 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b"} err="failed to get container status \"73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b\": rpc error: code = NotFound desc = could not find container \"73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b\": container with ID starting with 73b519fda79cb75e9233c80a84d4b6883c78d7a003ba9875a5260d10cc4d1d4b not found: ID does not exist" Jul 28 15:06:07.692301 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.692274 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27807ae5-c540-4a08-93a6-e6948a3c7510" path="/var/lib/kubelet/pods/27807ae5-c540-4a08-93a6-e6948a3c7510/volumes" Jul 28 15:06:07.692537 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.692525 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8"] Jul 28 15:06:07.692623 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:07.692566 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-84fa8d-68d84dd949-4cvw8"] Jul 28 15:06:09.691821 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:09.691789 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faeee89f-d121-446a-9a7e-c27fa4eba183" path="/var/lib/kubelet/pods/faeee89f-d121-446a-9a7e-c27fa4eba183/volumes" Jul 28 15:06:46.279973 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.279924 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2"] Jul 28 15:06:46.280515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.280359 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27807ae5-c540-4a08-93a6-e6948a3c7510" containerName="payload-processing" Jul 28 15:06:46.280515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.280374 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="27807ae5-c540-4a08-93a6-e6948a3c7510" containerName="payload-processing" Jul 28 15:06:46.280515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.280391 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faeee89f-d121-446a-9a7e-c27fa4eba183" containerName="maas-api" Jul 28 15:06:46.280515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.280397 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="faeee89f-d121-446a-9a7e-c27fa4eba183" containerName="maas-api" Jul 28 15:06:46.280515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.280484 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="faeee89f-d121-446a-9a7e-c27fa4eba183" containerName="maas-api" Jul 28 15:06:46.280515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.280495 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="27807ae5-c540-4a08-93a6-e6948a3c7510" containerName="payload-processing" Jul 28 15:06:46.283302 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.283286 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.285688 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.285666 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bc713f-serving-cert\"" Jul 28 15:06:46.285941 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.285925 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bk5gs\"" Jul 28 15:06:46.295147 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.295124 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2"] Jul 28 15:06:46.390139 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.390108 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.390306 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.390150 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnzp9\" (UniqueName: \"kubernetes.io/projected/8a664411-53e8-4bcc-8890-1c34c9381c17-kube-api-access-mnzp9\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.390306 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.390279 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a664411-53e8-4bcc-8890-1c34c9381c17-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.491070 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.491026 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a664411-53e8-4bcc-8890-1c34c9381c17-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.491257 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.491103 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.491257 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.491134 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnzp9\" (UniqueName: \"kubernetes.io/projected/8a664411-53e8-4bcc-8890-1c34c9381c17-kube-api-access-mnzp9\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.491257 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:06:46.491247 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-serving-cert: secret "maas-api-e2e-shared-b-bc713f-serving-cert" not found Jul 28 15:06:46.491383 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:06:46.491317 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls podName:8a664411-53e8-4bcc-8890-1c34c9381c17 nodeName:}" failed. No retries permitted until 2026-07-28 15:06:46.991298649 +0000 UTC m=+2637.792176199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls") pod "maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" (UID: "8a664411-53e8-4bcc-8890-1c34c9381c17") : secret "maas-api-e2e-shared-b-bc713f-serving-cert" not found Jul 28 15:06:46.491694 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.491662 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a664411-53e8-4bcc-8890-1c34c9381c17-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.499712 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.499688 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnzp9\" (UniqueName: \"kubernetes.io/projected/8a664411-53e8-4bcc-8890-1c34c9381c17-kube-api-access-mnzp9\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.995542 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.995502 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:46.998041 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:46.998013 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls\") pod \"maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:47.015412 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.015382 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5"] Jul 28 15:06:47.019317 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.019296 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.021756 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.021736 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-bc713f-dockercfg-t4z8g\"" Jul 28 15:06:47.021923 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.021909 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-bc713f\"" Jul 28 15:06:47.029614 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.029591 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5"] Jul 28 15:06:47.096519 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.096488 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47e5ab72-7947-42d6-9da5-8843e89d7145-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.096721 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.096544 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v79g9\" (UniqueName: \"kubernetes.io/projected/47e5ab72-7947-42d6-9da5-8843e89d7145-kube-api-access-v79g9\") pod \"payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.193821 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.193789 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:47.198129 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.198100 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47e5ab72-7947-42d6-9da5-8843e89d7145-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.198246 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.198144 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v79g9\" (UniqueName: \"kubernetes.io/projected/47e5ab72-7947-42d6-9da5-8843e89d7145-kube-api-access-v79g9\") pod \"payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.198853 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.198832 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47e5ab72-7947-42d6-9da5-8843e89d7145-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.207682 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.207654 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v79g9\" (UniqueName: \"kubernetes.io/projected/47e5ab72-7947-42d6-9da5-8843e89d7145-kube-api-access-v79g9\") pod \"payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.330933 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.330901 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2"] Jul 28 15:06:47.331821 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.331799 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.332178 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:06:47.332151 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a664411_53e8_4bcc_8890_1c34c9381c17.slice/crio-1d3248ec209579c1ed393c48a5f233e9f2d3e055ea6e20aa60a48b2c19ddaa71 WatchSource:0}: Error finding container 1d3248ec209579c1ed393c48a5f233e9f2d3e055ea6e20aa60a48b2c19ddaa71: Status 404 returned error can't find the container with id 1d3248ec209579c1ed393c48a5f233e9f2d3e055ea6e20aa60a48b2c19ddaa71 Jul 28 15:06:47.674476 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.674438 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5"] Jul 28 15:06:47.676341 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:06:47.676300 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e5ab72_7947_42d6_9da5_8843e89d7145.slice/crio-9d3cff03e8b39c1b499d78de84d949dfbbda03877172d99118cc9777850429ac WatchSource:0}: Error finding container 9d3cff03e8b39c1b499d78de84d949dfbbda03877172d99118cc9777850429ac: Status 404 returned error can't find the container with id 9d3cff03e8b39c1b499d78de84d949dfbbda03877172d99118cc9777850429ac Jul 28 15:06:47.896761 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.896728 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" event={"ID":"47e5ab72-7947-42d6-9da5-8843e89d7145","Type":"ContainerStarted","Data":"ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47"} Jul 28 15:06:47.896953 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.896791 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" event={"ID":"47e5ab72-7947-42d6-9da5-8843e89d7145","Type":"ContainerStarted","Data":"9d3cff03e8b39c1b499d78de84d949dfbbda03877172d99118cc9777850429ac"} Jul 28 15:06:47.896953 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.896818 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:06:47.898173 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.898151 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" event={"ID":"8a664411-53e8-4bcc-8890-1c34c9381c17","Type":"ContainerStarted","Data":"bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e"} Jul 28 15:06:47.898257 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.898191 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" event={"ID":"8a664411-53e8-4bcc-8890-1c34c9381c17","Type":"ContainerStarted","Data":"1d3248ec209579c1ed393c48a5f233e9f2d3e055ea6e20aa60a48b2c19ddaa71"} Jul 28 15:06:47.898257 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.898217 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:47.914030 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.913986 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" podStartSLOduration=1.913969606 podStartE2EDuration="1.913969606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:06:47.91134377 +0000 UTC m=+2638.712221326" watchObservedRunningTime="2026-07-28 15:06:47.913969606 +0000 UTC m=+2638.714847208" Jul 28 15:06:47.926637 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:47.926543 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" podStartSLOduration=1.552400171 podStartE2EDuration="1.926528499s" totalImagesPullingTime="374.128328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:06:47.925973906 +0000 UTC m=+2638.726851464" watchObservedRunningTime="2026-07-28 15:06:47.926528499 +0000 UTC m=+2638.727406072" Jul 28 15:06:53.909125 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:53.909097 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:06:58.905987 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:06:58.905957 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:16:25.975907 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:25.975861 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x2fkd"] Jul 28 15:16:25.979332 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:25.979316 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:25.983281 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:25.983253 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-fjb74\"" Jul 28 15:16:25.983450 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:25.983270 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 28 15:16:25.983836 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:25.983819 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 28 15:16:25.987374 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:25.987030 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x2fkd"] Jul 28 15:16:26.052759 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:26.052728 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl4pw\" (UniqueName: \"kubernetes.io/projected/c42f8aa9-817f-467e-a2a0-b446d858fbf2-kube-api-access-jl4pw\") pod \"kuadrant-operator-catalog-x2fkd\" (UID: \"c42f8aa9-817f-467e-a2a0-b446d858fbf2\") " pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:26.153560 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:26.153524 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4pw\" (UniqueName: \"kubernetes.io/projected/c42f8aa9-817f-467e-a2a0-b446d858fbf2-kube-api-access-jl4pw\") pod \"kuadrant-operator-catalog-x2fkd\" (UID: \"c42f8aa9-817f-467e-a2a0-b446d858fbf2\") " pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:26.162122 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:26.162097 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4pw\" (UniqueName: \"kubernetes.io/projected/c42f8aa9-817f-467e-a2a0-b446d858fbf2-kube-api-access-jl4pw\") pod \"kuadrant-operator-catalog-x2fkd\" (UID: \"c42f8aa9-817f-467e-a2a0-b446d858fbf2\") " pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:26.290539 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:26.290452 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:26.421838 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:26.421812 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x2fkd"] Jul 28 15:16:26.423443 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:16:26.423417 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42f8aa9_817f_467e_a2a0_b446d858fbf2.slice/crio-e87e07309e67ed905175b7593955982e89e694428bb42652a3fc17542a43dd1d WatchSource:0}: Error finding container e87e07309e67ed905175b7593955982e89e694428bb42652a3fc17542a43dd1d: Status 404 returned error can't find the container with id e87e07309e67ed905175b7593955982e89e694428bb42652a3fc17542a43dd1d Jul 28 15:16:26.424782 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:26.424765 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 15:16:27.109295 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:27.109261 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" event={"ID":"c42f8aa9-817f-467e-a2a0-b446d858fbf2","Type":"ContainerStarted","Data":"e87e07309e67ed905175b7593955982e89e694428bb42652a3fc17542a43dd1d"} Jul 28 15:16:29.124232 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:29.124197 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" event={"ID":"c42f8aa9-817f-467e-a2a0-b446d858fbf2","Type":"ContainerStarted","Data":"654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0"} Jul 28 15:16:29.141879 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:29.141831 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" podStartSLOduration=1.933307604 podStartE2EDuration="4.141814943s" totalImagesPullingTime="2.208507339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:16:29.138568945 +0000 UTC m=+3219.939446498" watchObservedRunningTime="2026-07-28 15:16:29.141814943 +0000 UTC m=+3219.942692498" Jul 28 15:16:36.290726 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:36.290680 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:36.291158 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:36.290749 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:36.312603 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:36.312571 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:37.192015 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:37.191983 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:37.226243 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:37.226210 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x2fkd"] Jul 28 15:16:39.181567 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:39.181517 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" podUID="c42f8aa9-817f-467e-a2a0-b446d858fbf2" containerName="registry-server" containerID="cri-o://654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0" gracePeriod=2 Jul 28 15:16:39.436425 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:39.436365 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:39.478452 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:39.478424 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl4pw\" (UniqueName: \"kubernetes.io/projected/c42f8aa9-817f-467e-a2a0-b446d858fbf2-kube-api-access-jl4pw\") pod \"c42f8aa9-817f-467e-a2a0-b446d858fbf2\" (UID: \"c42f8aa9-817f-467e-a2a0-b446d858fbf2\") " Jul 28 15:16:39.480487 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:39.480465 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42f8aa9-817f-467e-a2a0-b446d858fbf2-kube-api-access-jl4pw" (OuterVolumeSpecName: "kube-api-access-jl4pw") pod "c42f8aa9-817f-467e-a2a0-b446d858fbf2" (UID: "c42f8aa9-817f-467e-a2a0-b446d858fbf2"). InnerVolumeSpecName "kube-api-access-jl4pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:16:39.579519 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:39.579490 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl4pw\" (UniqueName: \"kubernetes.io/projected/c42f8aa9-817f-467e-a2a0-b446d858fbf2-kube-api-access-jl4pw\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:16:40.188130 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.188093 2586 generic.go:356] "Generic (PLEG): container finished" podID="c42f8aa9-817f-467e-a2a0-b446d858fbf2" containerID="654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0" exitCode=0 Jul 28 15:16:40.188533 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.188162 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" Jul 28 15:16:40.188533 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.188171 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" event={"ID":"c42f8aa9-817f-467e-a2a0-b446d858fbf2","Type":"ContainerDied","Data":"654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0"} Jul 28 15:16:40.188533 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.188250 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x2fkd" event={"ID":"c42f8aa9-817f-467e-a2a0-b446d858fbf2","Type":"ContainerDied","Data":"e87e07309e67ed905175b7593955982e89e694428bb42652a3fc17542a43dd1d"} Jul 28 15:16:40.188533 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.188270 2586 scope.go:117] "RemoveContainer" containerID="654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0" Jul 28 15:16:40.199082 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.199060 2586 scope.go:117] "RemoveContainer" containerID="654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0" Jul 28 15:16:40.199406 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:16:40.199387 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0\": container with ID starting with 654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0 not found: ID does not exist" containerID="654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0" Jul 28 15:16:40.199489 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.199418 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0"} err="failed to get container status \"654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0\": rpc error: code = NotFound desc = could not find container \"654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0\": container with ID starting with 654c29f8c073cb64d2106f041f83430d4d4c3f6995e65fef6241a9ba161acac0 not found: ID does not exist" Jul 28 15:16:40.220712 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.220683 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x2fkd"] Jul 28 15:16:40.228078 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:40.228054 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x2fkd"] Jul 28 15:16:41.691326 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:16:41.691251 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c42f8aa9-817f-467e-a2a0-b446d858fbf2" path="/var/lib/kubelet/pods/c42f8aa9-817f-467e-a2a0-b446d858fbf2/volumes" Jul 28 15:17:12.329332 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.329299 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p"] Jul 28 15:17:12.329926 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.329719 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42f8aa9-817f-467e-a2a0-b446d858fbf2" containerName="registry-server" Jul 28 15:17:12.329926 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.329731 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42f8aa9-817f-467e-a2a0-b446d858fbf2" containerName="registry-server" Jul 28 15:17:12.329926 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.329797 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42f8aa9-817f-467e-a2a0-b446d858fbf2" containerName="registry-server" Jul 28 15:17:12.333421 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.333402 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.335671 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.335650 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-86ef5d-serving-cert\"" Jul 28 15:17:12.344903 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.344877 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p"] Jul 28 15:17:12.494254 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.494219 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7hmh\" (UniqueName: \"kubernetes.io/projected/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-kube-api-access-g7hmh\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.494423 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.494281 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.494423 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.494346 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.595165 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.595075 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.595165 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.595133 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.595373 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.595199 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7hmh\" (UniqueName: \"kubernetes.io/projected/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-kube-api-access-g7hmh\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.595373 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:17:12.595227 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-serving-cert: secret "maas-api-e2e-ipp-b-86ef5d-serving-cert" not found Jul 28 15:17:12.595373 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:17:12.595296 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls podName:1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a nodeName:}" failed. No retries permitted until 2026-07-28 15:17:13.095280174 +0000 UTC m=+3263.896157711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls") pod "maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" (UID: "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a") : secret "maas-api-e2e-ipp-b-86ef5d-serving-cert" not found Jul 28 15:17:12.595909 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.595891 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:12.605197 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:12.605177 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7hmh\" (UniqueName: \"kubernetes.io/projected/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-kube-api-access-g7hmh\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:13.035071 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.034971 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg"] Jul 28 15:17:13.037465 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.037449 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.040905 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.040884 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-86ef5d\"" Jul 28 15:17:13.041297 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.041276 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-86ef5d-dockercfg-zmtxp\"" Jul 28 15:17:13.048675 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.048653 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg"] Jul 28 15:17:13.051925 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.051907 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8"] Jul 28 15:17:13.054226 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.054209 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.063062 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.063039 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8"] Jul 28 15:17:13.100421 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.100393 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:13.102833 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.102811 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:13.201223 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.201188 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vdt\" (UniqueName: \"kubernetes.io/projected/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-kube-api-access-z2vdt\") pod \"payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.201409 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.201268 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6bzr\" (UniqueName: \"kubernetes.io/projected/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-kube-api-access-l6bzr\") pod \"payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.201409 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.201313 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.201409 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.201350 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.245440 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.245412 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:13.302404 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.301996 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vdt\" (UniqueName: \"kubernetes.io/projected/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-kube-api-access-z2vdt\") pod \"payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.302544 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.302503 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6bzr\" (UniqueName: \"kubernetes.io/projected/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-kube-api-access-l6bzr\") pod \"payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.302635 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.302600 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.302697 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.302655 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.303670 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.303641 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.303823 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.303793 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.310782 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.310756 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vdt\" (UniqueName: \"kubernetes.io/projected/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-kube-api-access-z2vdt\") pod \"payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.310927 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.310902 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6bzr\" (UniqueName: \"kubernetes.io/projected/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-kube-api-access-l6bzr\") pod \"payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.348854 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.348826 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:13.365189 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.365161 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:13.386306 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.386281 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p"] Jul 28 15:17:13.387781 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:17:13.387748 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b27785e_cc6b_4ec3_bf7d_9a55b9e7979a.slice/crio-fa396ed0e1f9fc7b8f7f28dd2e19bc11e1cf1379bf84b785b86e27dd7f044c8a WatchSource:0}: Error finding container fa396ed0e1f9fc7b8f7f28dd2e19bc11e1cf1379bf84b785b86e27dd7f044c8a: Status 404 returned error can't find the container with id fa396ed0e1f9fc7b8f7f28dd2e19bc11e1cf1379bf84b785b86e27dd7f044c8a Jul 28 15:17:13.499851 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.499816 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg"] Jul 28 15:17:13.501998 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:17:13.501967 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad8ad1f_23b1_4b0c_a51c_51e6c7cefc09.slice/crio-9534beb121cf051d146f49cd672ead2db256e309ed7fd0f2f0810a899da1fade WatchSource:0}: Error finding container 9534beb121cf051d146f49cd672ead2db256e309ed7fd0f2f0810a899da1fade: Status 404 returned error can't find the container with id 9534beb121cf051d146f49cd672ead2db256e309ed7fd0f2f0810a899da1fade Jul 28 15:17:13.731450 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:13.731424 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8"] Jul 28 15:17:13.732634 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:17:13.732611 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce26f2f8_c582_46e6_8d28_b1f74b100ae5.slice/crio-b682d10a1316c1a9f09447aa8b58aa26e68cb5b956b277fadf3ffaeeece46ce0 WatchSource:0}: Error finding container b682d10a1316c1a9f09447aa8b58aa26e68cb5b956b277fadf3ffaeeece46ce0: Status 404 returned error can't find the container with id b682d10a1316c1a9f09447aa8b58aa26e68cb5b956b277fadf3ffaeeece46ce0 Jul 28 15:17:14.389176 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.389142 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" event={"ID":"ce26f2f8-c582-46e6-8d28-b1f74b100ae5","Type":"ContainerStarted","Data":"6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44"} Jul 28 15:17:14.389604 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.389205 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:14.389604 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.389230 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" event={"ID":"ce26f2f8-c582-46e6-8d28-b1f74b100ae5","Type":"ContainerStarted","Data":"b682d10a1316c1a9f09447aa8b58aa26e68cb5b956b277fadf3ffaeeece46ce0"} Jul 28 15:17:14.390696 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.390667 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" event={"ID":"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09","Type":"ContainerStarted","Data":"a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc"} Jul 28 15:17:14.390802 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.390726 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:14.390802 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.390744 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" event={"ID":"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09","Type":"ContainerStarted","Data":"9534beb121cf051d146f49cd672ead2db256e309ed7fd0f2f0810a899da1fade"} Jul 28 15:17:14.392197 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.392174 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" event={"ID":"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a","Type":"ContainerStarted","Data":"70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642"} Jul 28 15:17:14.392307 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.392219 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" event={"ID":"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a","Type":"ContainerStarted","Data":"fa396ed0e1f9fc7b8f7f28dd2e19bc11e1cf1379bf84b785b86e27dd7f044c8a"} Jul 28 15:17:14.392307 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.392255 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:14.404135 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.404023 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" podStartSLOduration=1.404010222 podStartE2EDuration="1.404010222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:17:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:17:14.403736483 +0000 UTC m=+3265.204614043" watchObservedRunningTime="2026-07-28 15:17:14.404010222 +0000 UTC m=+3265.204887839" Jul 28 15:17:14.421319 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.421265 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" podStartSLOduration=1.525310025 podStartE2EDuration="2.421251369s" totalImagesPullingTime="895.941344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:17:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:17:14.419079681 +0000 UTC m=+3265.219957237" watchObservedRunningTime="2026-07-28 15:17:14.421251369 +0000 UTC m=+3265.222128925" Jul 28 15:17:14.434010 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:14.433970 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" podStartSLOduration=1.433956357 podStartE2EDuration="1.433956357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 15:17:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:17:14.432690177 +0000 UTC m=+3265.233567734" watchObservedRunningTime="2026-07-28 15:17:14.433956357 +0000 UTC m=+3265.234833921" Jul 28 15:17:20.404533 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:20.404499 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:17:25.400699 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:25.400667 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:17:25.401159 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:25.400742 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:17:44.688808 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:17:44.688774 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8_ce26f2f8-c582-46e6-8d28-b1f74b100ae5/payload-processing/0.log" Jul 28 15:18:42.252670 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:18:42.252640 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8_ce26f2f8-c582-46e6-8d28-b1f74b100ae5/payload-processing/0.log" Jul 28 15:20:42.613223 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.613193 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p"] Jul 28 15:20:42.613718 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.613393 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" podUID="1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" containerName="maas-api" containerID="cri-o://70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642" gracePeriod=30 Jul 28 15:20:42.652905 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.652877 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8"] Jul 28 15:20:42.653143 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.653105 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" podUID="ce26f2f8-c582-46e6-8d28-b1f74b100ae5" containerName="payload-processing" containerID="cri-o://6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44" gracePeriod=30 Jul 28 15:20:42.660920 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.660891 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg"] Jul 28 15:20:42.661109 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.661089 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" podUID="3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" containerName="payload-pre-processing" containerID="cri-o://a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc" gracePeriod=30 Jul 28 15:20:42.923256 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.923231 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:20:42.986915 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.986894 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:20:42.989899 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:42.989880 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:20:43.100218 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100188 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-plugins-config-volume\") pod \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " Jul 28 15:20:43.100403 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100264 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6bzr\" (UniqueName: \"kubernetes.io/projected/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-kube-api-access-l6bzr\") pod \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\" (UID: \"ce26f2f8-c582-46e6-8d28-b1f74b100ae5\") " Jul 28 15:20:43.100403 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100306 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-plugins-config-volume\") pod \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " Jul 28 15:20:43.100403 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100336 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2vdt\" (UniqueName: \"kubernetes.io/projected/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-kube-api-access-z2vdt\") pod \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\" (UID: \"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09\") " Jul 28 15:20:43.100403 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100359 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-openshift-service-ca\") pod \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " Jul 28 15:20:43.100649 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100436 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7hmh\" (UniqueName: \"kubernetes.io/projected/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-kube-api-access-g7hmh\") pod \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " Jul 28 15:20:43.100649 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100465 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls\") pod \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\" (UID: \"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a\") " Jul 28 15:20:43.100767 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100715 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce26f2f8-c582-46e6-8d28-b1f74b100ae5" (UID: "ce26f2f8-c582-46e6-8d28-b1f74b100ae5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:20:43.100819 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100788 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" (UID: "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:20:43.100908 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100888 2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-plugins-config-volume\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.100966 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100910 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-openshift-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.101019 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.100967 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" (UID: "3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:20:43.102464 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.102432 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-kube-api-access-l6bzr" (OuterVolumeSpecName: "kube-api-access-l6bzr") pod "ce26f2f8-c582-46e6-8d28-b1f74b100ae5" (UID: "ce26f2f8-c582-46e6-8d28-b1f74b100ae5"). InnerVolumeSpecName "kube-api-access-l6bzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:20:43.102672 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.102507 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" (UID: "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 15:20:43.102672 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.102592 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-kube-api-access-z2vdt" (OuterVolumeSpecName: "kube-api-access-z2vdt") pod "3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" (UID: "3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09"). InnerVolumeSpecName "kube-api-access-z2vdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:20:43.102799 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.102730 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-kube-api-access-g7hmh" (OuterVolumeSpecName: "kube-api-access-g7hmh") pod "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" (UID: "1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a"). InnerVolumeSpecName "kube-api-access-g7hmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:20:43.201952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.201867 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7hmh\" (UniqueName: \"kubernetes.io/projected/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-kube-api-access-g7hmh\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.201952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.201898 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a-maas-api-tls\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.201952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.201908 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6bzr\" (UniqueName: \"kubernetes.io/projected/ce26f2f8-c582-46e6-8d28-b1f74b100ae5-kube-api-access-l6bzr\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.201952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.201916 2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-plugins-config-volume\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.201952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.201924 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2vdt\" (UniqueName: \"kubernetes.io/projected/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09-kube-api-access-z2vdt\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:20:43.578515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.578476 2586 generic.go:356] "Generic (PLEG): container finished" podID="1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" containerID="70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642" exitCode=0 Jul 28 15:20:43.578701 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.578526 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" Jul 28 15:20:43.578701 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.578582 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" event={"ID":"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a","Type":"ContainerDied","Data":"70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642"} Jul 28 15:20:43.578701 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.578634 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p" event={"ID":"1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a","Type":"ContainerDied","Data":"fa396ed0e1f9fc7b8f7f28dd2e19bc11e1cf1379bf84b785b86e27dd7f044c8a"} Jul 28 15:20:43.578701 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.578661 2586 scope.go:117] "RemoveContainer" containerID="70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642" Jul 28 15:20:43.579859 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.579840 2586 generic.go:356] "Generic (PLEG): container finished" podID="ce26f2f8-c582-46e6-8d28-b1f74b100ae5" containerID="6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44" exitCode=0 Jul 28 15:20:43.579948 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.579881 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" Jul 28 15:20:43.580007 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.579874 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" event={"ID":"ce26f2f8-c582-46e6-8d28-b1f74b100ae5","Type":"ContainerDied","Data":"6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44"} Jul 28 15:20:43.580067 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.580010 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8" event={"ID":"ce26f2f8-c582-46e6-8d28-b1f74b100ae5","Type":"ContainerDied","Data":"b682d10a1316c1a9f09447aa8b58aa26e68cb5b956b277fadf3ffaeeece46ce0"} Jul 28 15:20:43.581283 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.581260 2586 generic.go:356] "Generic (PLEG): container finished" podID="3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" containerID="a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc" exitCode=0 Jul 28 15:20:43.581378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.581280 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" event={"ID":"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09","Type":"ContainerDied","Data":"a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc"} Jul 28 15:20:43.581378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.581315 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" Jul 28 15:20:43.581378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.581325 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg" event={"ID":"3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09","Type":"ContainerDied","Data":"9534beb121cf051d146f49cd672ead2db256e309ed7fd0f2f0810a899da1fade"} Jul 28 15:20:43.590469 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.590452 2586 scope.go:117] "RemoveContainer" containerID="70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642" Jul 28 15:20:43.590761 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:20:43.590740 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642\": container with ID starting with 70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642 not found: ID does not exist" containerID="70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642" Jul 28 15:20:43.590832 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.590762 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642"} err="failed to get container status \"70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642\": rpc error: code = NotFound desc = could not find container \"70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642\": container with ID starting with 70f40682a59da3aa4a0524ba2affef7710afa473d3f177f7bcc32685ba1da642 not found: ID does not exist" Jul 28 15:20:43.590832 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.590778 2586 scope.go:117] "RemoveContainer" containerID="6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44" Jul 28 15:20:43.599261 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.599246 2586 scope.go:117] "RemoveContainer" containerID="6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44" Jul 28 15:20:43.599493 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:20:43.599478 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44\": container with ID starting with 6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44 not found: ID does not exist" containerID="6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44" Jul 28 15:20:43.599541 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.599499 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44"} err="failed to get container status \"6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44\": rpc error: code = NotFound desc = could not find container \"6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44\": container with ID starting with 6b589768221886192930cb0eda25cdc5727bb0d0b1a9000003e9234326fadf44 not found: ID does not exist" Jul 28 15:20:43.599541 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.599514 2586 scope.go:117] "RemoveContainer" containerID="a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc" Jul 28 15:20:43.603783 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.603766 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p"] Jul 28 15:20:43.606408 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.606388 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86ef5d-7996cbbbb5-7r82p"] Jul 28 15:20:43.608094 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.608082 2586 scope.go:117] "RemoveContainer" containerID="a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc" Jul 28 15:20:43.608331 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:20:43.608316 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc\": container with ID starting with a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc not found: ID does not exist" containerID="a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc" Jul 28 15:20:43.608378 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.608336 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc"} err="failed to get container status \"a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc\": rpc error: code = NotFound desc = could not find container \"a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc\": container with ID starting with a4a3f0ddf1456ae671b8e697f9b7845903e8b51b18790eda642432a2c6e731cc not found: ID does not exist" Jul 28 15:20:43.614540 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.614521 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg"] Jul 28 15:20:43.621120 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.621101 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-86ef5d-bd7f78668-dg5gg"] Jul 28 15:20:43.630600 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.630580 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8"] Jul 28 15:20:43.632739 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.632721 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86ef5d-fbd96b8bd-9jtc8"] Jul 28 15:20:43.691945 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.691911 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" path="/var/lib/kubelet/pods/1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a/volumes" Jul 28 15:20:43.692285 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.692272 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" path="/var/lib/kubelet/pods/3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09/volumes" Jul 28 15:20:43.692592 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:20:43.692581 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce26f2f8-c582-46e6-8d28-b1f74b100ae5" path="/var/lib/kubelet/pods/ce26f2f8-c582-46e6-8d28-b1f74b100ae5/volumes" Jul 28 15:21:18.081701 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.081614 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2"] Jul 28 15:21:18.082289 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.081824 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" podUID="8a664411-53e8-4bcc-8890-1c34c9381c17" containerName="maas-api" containerID="cri-o://bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e" gracePeriod=30 Jul 28 15:21:18.139254 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.139217 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5"] Jul 28 15:21:18.139488 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.139461 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" podUID="47e5ab72-7947-42d6-9da5-8843e89d7145" containerName="payload-processing" containerID="cri-o://ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47" gracePeriod=30 Jul 28 15:21:18.394128 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.394103 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:21:18.419578 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.419528 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:21:18.529763 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.529718 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v79g9\" (UniqueName: \"kubernetes.io/projected/47e5ab72-7947-42d6-9da5-8843e89d7145-kube-api-access-v79g9\") pod \"47e5ab72-7947-42d6-9da5-8843e89d7145\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " Jul 28 15:21:18.529955 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.529789 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47e5ab72-7947-42d6-9da5-8843e89d7145-plugins-config-volume\") pod \"47e5ab72-7947-42d6-9da5-8843e89d7145\" (UID: \"47e5ab72-7947-42d6-9da5-8843e89d7145\") " Jul 28 15:21:18.529955 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.529845 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnzp9\" (UniqueName: \"kubernetes.io/projected/8a664411-53e8-4bcc-8890-1c34c9381c17-kube-api-access-mnzp9\") pod \"8a664411-53e8-4bcc-8890-1c34c9381c17\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " Jul 28 15:21:18.529955 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.529906 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls\") pod \"8a664411-53e8-4bcc-8890-1c34c9381c17\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " Jul 28 15:21:18.529955 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.529947 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a664411-53e8-4bcc-8890-1c34c9381c17-openshift-service-ca\") pod \"8a664411-53e8-4bcc-8890-1c34c9381c17\" (UID: \"8a664411-53e8-4bcc-8890-1c34c9381c17\") " Jul 28 15:21:18.530163 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.530141 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e5ab72-7947-42d6-9da5-8843e89d7145-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47e5ab72-7947-42d6-9da5-8843e89d7145" (UID: "47e5ab72-7947-42d6-9da5-8843e89d7145"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:21:18.530354 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.530329 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a664411-53e8-4bcc-8890-1c34c9381c17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a664411-53e8-4bcc-8890-1c34c9381c17" (UID: "8a664411-53e8-4bcc-8890-1c34c9381c17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 15:21:18.530488 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.530463 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a664411-53e8-4bcc-8890-1c34c9381c17-openshift-service-ca\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:21:18.530570 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.530492 2586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47e5ab72-7947-42d6-9da5-8843e89d7145-plugins-config-volume\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:21:18.531925 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.531902 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a664411-53e8-4bcc-8890-1c34c9381c17" (UID: "8a664411-53e8-4bcc-8890-1c34c9381c17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 15:21:18.532028 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.531924 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e5ab72-7947-42d6-9da5-8843e89d7145-kube-api-access-v79g9" (OuterVolumeSpecName: "kube-api-access-v79g9") pod "47e5ab72-7947-42d6-9da5-8843e89d7145" (UID: "47e5ab72-7947-42d6-9da5-8843e89d7145"). InnerVolumeSpecName "kube-api-access-v79g9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:21:18.532028 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.531971 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a664411-53e8-4bcc-8890-1c34c9381c17-kube-api-access-mnzp9" (OuterVolumeSpecName: "kube-api-access-mnzp9") pod "8a664411-53e8-4bcc-8890-1c34c9381c17" (UID: "8a664411-53e8-4bcc-8890-1c34c9381c17"). InnerVolumeSpecName "kube-api-access-mnzp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 15:21:18.632004 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.631915 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnzp9\" (UniqueName: \"kubernetes.io/projected/8a664411-53e8-4bcc-8890-1c34c9381c17-kube-api-access-mnzp9\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:21:18.632004 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.631957 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a664411-53e8-4bcc-8890-1c34c9381c17-maas-api-tls\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:21:18.632004 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.631969 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v79g9\" (UniqueName: \"kubernetes.io/projected/47e5ab72-7947-42d6-9da5-8843e89d7145-kube-api-access-v79g9\") on node \"ip-10-0-131-88.ec2.internal\" DevicePath \"\"" Jul 28 15:21:18.776465 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.776433 2586 generic.go:356] "Generic (PLEG): container finished" podID="8a664411-53e8-4bcc-8890-1c34c9381c17" containerID="bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e" exitCode=0 Jul 28 15:21:18.776658 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.776478 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" Jul 28 15:21:18.776658 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.776518 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" event={"ID":"8a664411-53e8-4bcc-8890-1c34c9381c17","Type":"ContainerDied","Data":"bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e"} Jul 28 15:21:18.776658 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.776596 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2" event={"ID":"8a664411-53e8-4bcc-8890-1c34c9381c17","Type":"ContainerDied","Data":"1d3248ec209579c1ed393c48a5f233e9f2d3e055ea6e20aa60a48b2c19ddaa71"} Jul 28 15:21:18.776658 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.776626 2586 scope.go:117] "RemoveContainer" containerID="bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e" Jul 28 15:21:18.777876 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.777851 2586 generic.go:356] "Generic (PLEG): container finished" podID="47e5ab72-7947-42d6-9da5-8843e89d7145" containerID="ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47" exitCode=0 Jul 28 15:21:18.777972 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.777894 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" Jul 28 15:21:18.777972 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.777896 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" event={"ID":"47e5ab72-7947-42d6-9da5-8843e89d7145","Type":"ContainerDied","Data":"ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47"} Jul 28 15:21:18.777972 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.777943 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5" event={"ID":"47e5ab72-7947-42d6-9da5-8843e89d7145","Type":"ContainerDied","Data":"9d3cff03e8b39c1b499d78de84d949dfbbda03877172d99118cc9777850429ac"} Jul 28 15:21:18.787324 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.787310 2586 scope.go:117] "RemoveContainer" containerID="bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e" Jul 28 15:21:18.787659 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:21:18.787617 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e\": container with ID starting with bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e not found: ID does not exist" containerID="bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e" Jul 28 15:21:18.787812 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.787655 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e"} err="failed to get container status \"bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e\": rpc error: code = NotFound desc = could not find container \"bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e\": container with ID starting with bb7cec18c178c0587b407839442966ab4bb021615dab5cba46cb9cf6663f1a4e not found: ID does not exist" Jul 28 15:21:18.787812 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.787678 2586 scope.go:117] "RemoveContainer" containerID="ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47" Jul 28 15:21:18.795961 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.795943 2586 scope.go:117] "RemoveContainer" containerID="ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47" Jul 28 15:21:18.796225 ip-10-0-131-88 kubenswrapper[2586]: E0728 15:21:18.796209 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47\": container with ID starting with ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47 not found: ID does not exist" containerID="ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47" Jul 28 15:21:18.796272 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.796230 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47"} err="failed to get container status \"ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47\": rpc error: code = NotFound desc = could not find container \"ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47\": container with ID starting with ede4d810c9a221ac139a8e5ebba516a2ca83b2edd353197ee36fb268b47ecd47 not found: ID does not exist" Jul 28 15:21:18.816997 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.816973 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5"] Jul 28 15:21:18.821311 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.821290 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bc713f-599b77f67d-2vxj5"] Jul 28 15:21:18.838280 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.838254 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2"] Jul 28 15:21:18.846499 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:18.846467 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bc713f-7d4cdc9dfc-cf5j2"] Jul 28 15:21:19.691309 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:19.691272 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47e5ab72-7947-42d6-9da5-8843e89d7145" path="/var/lib/kubelet/pods/47e5ab72-7947-42d6-9da5-8843e89d7145/volumes" Jul 28 15:21:19.691692 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:19.691678 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a664411-53e8-4bcc-8890-1c34c9381c17" path="/var/lib/kubelet/pods/8a664411-53e8-4bcc-8890-1c34c9381c17/volumes" Jul 28 15:21:32.024868 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:32.024837 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f457fb848-bv5sc_818d3443-47a8-4e07-a841-77ce24331a17/copy-manifests/0.log" Jul 28 15:21:32.031835 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:32.031810 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f457fb848-bv5sc_818d3443-47a8-4e07-a841-77ce24331a17/manager/0.log" Jul 28 15:21:32.650463 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:32.650435 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-569f669744-tlbqv_cccd086d-f697-4fc1-98bc-dcced5390d70/maas-api/0.log" Jul 28 15:21:33.907752 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:33.907723 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss_10ffdfab-cdaa-48b8-85b0-f5947fe34917/util/0.log" Jul 28 15:21:33.913343 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:33.913322 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss_10ffdfab-cdaa-48b8-85b0-f5947fe34917/pull/0.log" Jul 28 15:21:33.918781 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:33.918764 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss_10ffdfab-cdaa-48b8-85b0-f5947fe34917/extract/0.log" Jul 28 15:21:34.026993 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.026960 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w_e4578db9-400c-4e01-93ef-4e11b2ecfb71/util/0.log" Jul 28 15:21:34.032767 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.032745 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w_e4578db9-400c-4e01-93ef-4e11b2ecfb71/pull/0.log" Jul 28 15:21:34.038473 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.038458 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w_e4578db9-400c-4e01-93ef-4e11b2ecfb71/extract/0.log" Jul 28 15:21:34.148719 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.148685 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r_f309ddcd-2cdf-45dc-bc32-0701a4e0d282/util/0.log" Jul 28 15:21:34.155247 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.155226 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r_f309ddcd-2cdf-45dc-bc32-0701a4e0d282/pull/0.log" Jul 28 15:21:34.161440 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.161391 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r_f309ddcd-2cdf-45dc-bc32-0701a4e0d282/extract/0.log" Jul 28 15:21:34.279436 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.279406 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw_fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4/util/0.log" Jul 28 15:21:34.285803 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.285780 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw_fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4/pull/0.log" Jul 28 15:21:34.291883 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:34.291867 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw_fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4/extract/0.log" Jul 28 15:21:35.600132 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:35.600108 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6_16174b3f-7b3a-4841-9f09-34edcb19934d/istio-proxy/0.log" Jul 28 15:21:35.728112 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:35.728082 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-74jxg_0b16d15a-cf1a-4ee7-9240-b9d813ba546a/discovery/0.log" Jul 28 15:21:36.190619 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:36.190585 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-b8c9t_3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4/payload-pre-processing/0.log" Jul 28 15:21:36.873231 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:36.873203 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6hz8d_7be827ab-615b-4e18-93fd-2d672917c2d1/main/0.log" Jul 28 15:21:36.982878 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:36.982852 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mxppb_0d6522c1-7dbc-4f25-9e49-8e7380978b71/main/0.log" Jul 28 15:21:46.504825 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:46.504793 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-69ppv_f23822e0-3128-490e-933b-af0a29a96495/global-pull-secret-syncer/0.log" Jul 28 15:21:46.622450 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:46.622417 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rj94j_771b4120-696d-441e-b6de-22ceb421fe5e/konnectivity-agent/0.log" Jul 28 15:21:46.683141 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:46.683110 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-88.ec2.internal_6ff9d18249fb2db5835b1f8658874702/haproxy/0.log" Jul 28 15:21:50.551992 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.551911 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss_10ffdfab-cdaa-48b8-85b0-f5947fe34917/extract/0.log" Jul 28 15:21:50.573150 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.573124 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss_10ffdfab-cdaa-48b8-85b0-f5947fe34917/util/0.log" Jul 28 15:21:50.598855 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.598827 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pcvss_10ffdfab-cdaa-48b8-85b0-f5947fe34917/pull/0.log" Jul 28 15:21:50.627783 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.627758 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w_e4578db9-400c-4e01-93ef-4e11b2ecfb71/extract/0.log" Jul 28 15:21:50.649826 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.649797 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w_e4578db9-400c-4e01-93ef-4e11b2ecfb71/util/0.log" Jul 28 15:21:50.673881 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.673855 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xz79w_e4578db9-400c-4e01-93ef-4e11b2ecfb71/pull/0.log" Jul 28 15:21:50.703884 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.703860 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r_f309ddcd-2cdf-45dc-bc32-0701a4e0d282/extract/0.log" Jul 28 15:21:50.732458 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.732431 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r_f309ddcd-2cdf-45dc-bc32-0701a4e0d282/util/0.log" Jul 28 15:21:50.760774 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.760749 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732h27r_f309ddcd-2cdf-45dc-bc32-0701a4e0d282/pull/0.log" Jul 28 15:21:50.793079 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.793057 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw_fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4/extract/0.log" Jul 28 15:21:50.819450 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.819424 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw_fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4/util/0.log" Jul 28 15:21:50.838900 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:50.838877 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1qlbvw_fc9a001b-91a5-46d1-b90c-2d5cbee6f5a4/pull/0.log" Jul 28 15:21:53.173978 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.173945 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/alertmanager/0.log" Jul 28 15:21:53.202463 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.202435 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/config-reloader/0.log" Jul 28 15:21:53.223071 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.223043 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/kube-rbac-proxy-web/0.log" Jul 28 15:21:53.242178 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.242151 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/kube-rbac-proxy/0.log" Jul 28 15:21:53.261757 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.261730 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/kube-rbac-proxy-metric/0.log" Jul 28 15:21:53.303131 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.303103 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/prom-label-proxy/0.log" Jul 28 15:21:53.325770 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.325750 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_70f08444-cbcc-4825-8a53-6c4fe6f2ad37/init-config-reloader/0.log" Jul 28 15:21:53.372035 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.372004 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-wg8ns_139aebeb-aaf4-4581-96fd-a0ea5fc69b32/cluster-monitoring-operator/0.log" Jul 28 15:21:53.481241 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.481170 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7469fb649c-dhfdp_5aaefdc4-e72b-419e-9d4a-7c89c50aa3a4/metrics-server/0.log" Jul 28 15:21:53.508594 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.508568 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-2md2j_684319de-63bb-4f34-bd35-734c0fcd8b2a/monitoring-plugin/0.log" Jul 28 15:21:53.615515 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.615486 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n7zg9_50021748-9356-4992-a3ce-fc064f6e02b0/node-exporter/0.log" Jul 28 15:21:53.639919 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.639894 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n7zg9_50021748-9356-4992-a3ce-fc064f6e02b0/kube-rbac-proxy/0.log" Jul 28 15:21:53.660883 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.660860 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n7zg9_50021748-9356-4992-a3ce-fc064f6e02b0/init-textfile/0.log" Jul 28 15:21:53.791331 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.791245 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-nmzp2_91c33a8e-5d40-443d-996a-62a489c2e5e5/kube-rbac-proxy-main/0.log" Jul 28 15:21:53.812652 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.812618 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-nmzp2_91c33a8e-5d40-443d-996a-62a489c2e5e5/kube-rbac-proxy-self/0.log" Jul 28 15:21:53.832587 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:53.832544 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-nmzp2_91c33a8e-5d40-443d-996a-62a489c2e5e5/openshift-state-metrics/0.log" Jul 28 15:21:54.132218 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.132180 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-zv6f5_c54539fe-2711-4fcd-ac5e-59d4922447ed/prometheus-operator-admission-webhook/0.log" Jul 28 15:21:54.179207 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.179177 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d7fbdd4f8-vss6s_8eadbf34-39b7-4208-9efd-1b8f3b00129b/telemeter-client/0.log" Jul 28 15:21:54.212037 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.212004 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d7fbdd4f8-vss6s_8eadbf34-39b7-4208-9efd-1b8f3b00129b/reload/0.log" Jul 28 15:21:54.234971 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.234939 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d7fbdd4f8-vss6s_8eadbf34-39b7-4208-9efd-1b8f3b00129b/kube-rbac-proxy/0.log" Jul 28 15:21:54.266853 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.266818 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f946987f7-jcj4h_6bedff0c-1136-4a44-a5ef-c88aed387a56/thanos-query/0.log" Jul 28 15:21:54.286150 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.286129 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f946987f7-jcj4h_6bedff0c-1136-4a44-a5ef-c88aed387a56/kube-rbac-proxy-web/0.log" Jul 28 15:21:54.311306 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.311279 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f946987f7-jcj4h_6bedff0c-1136-4a44-a5ef-c88aed387a56/kube-rbac-proxy/0.log" Jul 28 15:21:54.327794 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.327768 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f946987f7-jcj4h_6bedff0c-1136-4a44-a5ef-c88aed387a56/prom-label-proxy/0.log" Jul 28 15:21:54.346652 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.346625 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f946987f7-jcj4h_6bedff0c-1136-4a44-a5ef-c88aed387a56/kube-rbac-proxy-rules/0.log" Jul 28 15:21:54.366502 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:54.366470 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f946987f7-jcj4h_6bedff0c-1136-4a44-a5ef-c88aed387a56/kube-rbac-proxy-metrics/0.log" Jul 28 15:21:56.975876 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:56.975851 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f6686b5f6-5xpqt_ed0395d3-4932-4dc1-b345-5c231f384eaa/console/0.log" Jul 28 15:21:57.007730 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.007701 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-chxst_4770e716-7472-4083-a75a-35a78ada6fb4/download-server/0.log" Jul 28 15:21:57.698288 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.698261 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-n4m4d_024aa77d-658a-45b4-8a26-eca38899c269/volume-data-source-validator/0.log" Jul 28 15:21:57.757894 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.757858 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q"] Jul 28 15:21:57.758264 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758252 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e5ab72-7947-42d6-9da5-8843e89d7145" containerName="payload-processing" Jul 28 15:21:57.758311 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758266 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e5ab72-7947-42d6-9da5-8843e89d7145" containerName="payload-processing" Jul 28 15:21:57.758311 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758283 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce26f2f8-c582-46e6-8d28-b1f74b100ae5" containerName="payload-processing" Jul 28 15:21:57.758311 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758289 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce26f2f8-c582-46e6-8d28-b1f74b100ae5" containerName="payload-processing" Jul 28 15:21:57.758311 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758299 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" containerName="maas-api" Jul 28 15:21:57.758311 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758306 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" containerName="maas-api" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758316 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a664411-53e8-4bcc-8890-1c34c9381c17" containerName="maas-api" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758321 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a664411-53e8-4bcc-8890-1c34c9381c17" containerName="maas-api" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758337 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" containerName="payload-pre-processing" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758342 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" containerName="payload-pre-processing" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758396 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e5ab72-7947-42d6-9da5-8843e89d7145" containerName="payload-processing" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758403 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce26f2f8-c582-46e6-8d28-b1f74b100ae5" containerName="payload-processing" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758410 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a664411-53e8-4bcc-8890-1c34c9381c17" containerName="maas-api" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758418 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad8ad1f-23b1-4b0c-a51c-51e6c7cefc09" containerName="payload-pre-processing" Jul 28 15:21:57.758460 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.758424 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b27785e-cc6b-4ec3-bf7d-9a55b9e7979a" containerName="maas-api" Jul 28 15:21:57.761032 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.761017 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:57.763317 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.763288 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9w7l5\"/\"kube-root-ca.crt\"" Jul 28 15:21:57.763420 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.763392 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9w7l5\"/\"default-dockercfg-krzlf\"" Jul 28 15:21:57.763420 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.763406 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9w7l5\"/\"openshift-service-ca.crt\"" Jul 28 15:21:57.768522 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.768497 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q"] Jul 28 15:21:57.919533 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.919495 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g89xk\" (UniqueName: \"kubernetes.io/projected/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-kube-api-access-g89xk\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:57.919711 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.919572 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-proc\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:57.919711 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.919625 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-podres\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:57.919711 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.919684 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-sys\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:57.919820 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:57.919737 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-lib-modules\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020395 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020302 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g89xk\" (UniqueName: \"kubernetes.io/projected/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-kube-api-access-g89xk\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020395 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020362 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-proc\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020395 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020399 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-podres\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020888 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020424 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-sys\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020888 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020454 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-lib-modules\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020888 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020471 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-proc\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020888 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020574 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-sys\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020888 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020597 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-podres\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.020888 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.020602 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-lib-modules\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.028160 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.028136 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g89xk\" (UniqueName: \"kubernetes.io/projected/d1bb698e-8f2a-495f-bba4-eb95ea3ea307-kube-api-access-g89xk\") pod \"perf-node-gather-daemonset-dlp2q\" (UID: \"d1bb698e-8f2a-495f-bba4-eb95ea3ea307\") " pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.073077 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.073045 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:58.205010 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.204983 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q"] Jul 28 15:21:58.206901 ip-10-0-131-88 kubenswrapper[2586]: W0728 15:21:58.206875 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd1bb698e_8f2a_495f_bba4_eb95ea3ea307.slice/crio-eaa635d235ca613109784321123c6a8dd2093a714f8a18c951c4ead7a7b5154e WatchSource:0}: Error finding container eaa635d235ca613109784321123c6a8dd2093a714f8a18c951c4ead7a7b5154e: Status 404 returned error can't find the container with id eaa635d235ca613109784321123c6a8dd2093a714f8a18c951c4ead7a7b5154e Jul 28 15:21:58.900961 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.900917 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fkf52_963b1cde-5993-4b5f-b189-88240de3db8e/dns/0.log" Jul 28 15:21:58.920482 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.920455 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fkf52_963b1cde-5993-4b5f-b189-88240de3db8e/kube-rbac-proxy/0.log" Jul 28 15:21:58.987454 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:58.987418 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cllq2_2df07b58-919e-42f6-a7f8-f9d18088cffd/dns-node-resolver/0.log" Jul 28 15:21:59.006735 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:59.006702 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" event={"ID":"d1bb698e-8f2a-495f-bba4-eb95ea3ea307","Type":"ContainerStarted","Data":"5c1cd5ff6eae86467e760c27736e5ed3cfa25ee66c240a90be5da343748bb6b8"} Jul 28 15:21:59.006906 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:59.006752 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" event={"ID":"d1bb698e-8f2a-495f-bba4-eb95ea3ea307","Type":"ContainerStarted","Data":"eaa635d235ca613109784321123c6a8dd2093a714f8a18c951c4ead7a7b5154e"} Jul 28 15:21:59.006906 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:59.006811 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:21:59.022622 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:59.022545 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" podStartSLOduration=2.02252847 podStartE2EDuration="2.02252847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 15:21:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 15:21:59.022137572 +0000 UTC m=+3549.823015130" watchObservedRunningTime="2026-07-28 15:21:59.02252847 +0000 UTC m=+3549.823406026" Jul 28 15:21:59.558785 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:59.558741 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5945b9bd66-2dr6h_9ca68c23-1bc0-447e-96b9-b236a7fe8c41/registry/0.log" Jul 28 15:21:59.600924 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:21:59.600900 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h7255_97c6451f-7a7a-41c6-bbe6-fa6d6733fb05/node-ca/0.log" Jul 28 15:22:00.557308 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:00.557270 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6_16174b3f-7b3a-4841-9f09-34edcb19934d/istio-proxy/0.log" Jul 28 15:22:00.707183 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:00.707146 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-74jxg_0b16d15a-cf1a-4ee7-9240-b9d813ba546a/discovery/0.log" Jul 28 15:22:00.829362 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:00.829267 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-b8c9t_3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4/payload-pre-processing/0.log" Jul 28 15:22:01.678687 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:01.678642 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hcw5f_4d8e0366-3dbf-4823-9eab-474ae1a2b4a8/serve-healthcheck-canary/0.log" Jul 28 15:22:02.270264 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:02.270233 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzqmb_149e8a6d-4f81-4b73-835f-9860a0d79164/kube-rbac-proxy/0.log" Jul 28 15:22:02.289207 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:02.289182 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzqmb_149e8a6d-4f81-4b73-835f-9860a0d79164/exporter/0.log" Jul 28 15:22:02.311063 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:02.311034 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzqmb_149e8a6d-4f81-4b73-835f-9860a0d79164/extractor/0.log" Jul 28 15:22:04.973750 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:04.973720 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f457fb848-bv5sc_818d3443-47a8-4e07-a841-77ce24331a17/manager/0.log" Jul 28 15:22:04.993521 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:04.993496 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f457fb848-bv5sc_818d3443-47a8-4e07-a841-77ce24331a17/copy-manifests/0.log" Jul 28 15:22:05.022698 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:05.022645 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9w7l5/perf-node-gather-daemonset-dlp2q" Jul 28 15:22:06.327674 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:06.327643 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7cf745d86f-ckhds_5df1bc09-634a-4bac-8917-6e2a20aefe19/manager/0.log" Jul 28 15:22:06.354035 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:06.354009 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gvhf6_d791c6a0-0447-47e5-a606-33c0f21d5c80/openshift-lws-operator/0.log" Jul 28 15:22:07.093952 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:07.093913 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6_16174b3f-7b3a-4841-9f09-34edcb19934d/istio-proxy/0.log" Jul 28 15:22:07.237573 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:07.237533 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-74jxg_0b16d15a-cf1a-4ee7-9240-b9d813ba546a/discovery/0.log" Jul 28 15:22:07.333981 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:07.333952 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-b8c9t_3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4/payload-pre-processing/0.log" Jul 28 15:22:08.808316 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:08.808288 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-w7ms6_16174b3f-7b3a-4841-9f09-34edcb19934d/istio-proxy/0.log" Jul 28 15:22:08.958000 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:08.957966 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-74jxg_0b16d15a-cf1a-4ee7-9240-b9d813ba546a/discovery/0.log" Jul 28 15:22:09.054930 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:09.054897 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-b8c9t_3ab2eb85-0d1e-4fc6-be67-efb3f38aafd4/payload-pre-processing/0.log" Jul 28 15:22:12.073362 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:12.073332 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss_b9551376-11dc-4d39-8d3d-ef0abd3d5047/kube-storage-version-migrator-operator/1.log" Jul 28 15:22:12.074383 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:12.074362 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-xgcss_b9551376-11dc-4d39-8d3d-ef0abd3d5047/kube-storage-version-migrator-operator/0.log" Jul 28 15:22:13.133516 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.133488 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-45bhv_158a522e-afdc-402a-9f80-1de157fcfd83/kube-multus/0.log" Jul 28 15:22:13.442941 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.442864 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/kube-multus-additional-cni-plugins/0.log" Jul 28 15:22:13.461906 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.461881 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/egress-router-binary-copy/0.log" Jul 28 15:22:13.480828 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.480803 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/cni-plugins/0.log" Jul 28 15:22:13.499233 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.499210 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/bond-cni-plugin/0.log" Jul 28 15:22:13.518598 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.518575 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/routeoverride-cni/0.log" Jul 28 15:22:13.537269 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.537243 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/whereabouts-cni-bincopy/0.log" Jul 28 15:22:13.556855 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.556833 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xkwwc_58f9d7a9-55e8-4acf-8909-a946ed151c2e/whereabouts-cni/0.log" Jul 28 15:22:13.653345 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.653314 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gcnr_750f2fca-ecd9-498a-babe-4e519ca4e8eb/network-metrics-daemon/0.log" Jul 28 15:22:13.669792 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:13.669767 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gcnr_750f2fca-ecd9-498a-babe-4e519ca4e8eb/kube-rbac-proxy/0.log" Jul 28 15:22:14.744074 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.744034 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/ovn-controller/0.log" Jul 28 15:22:14.763218 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.763184 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/ovn-acl-logging/0.log" Jul 28 15:22:14.784277 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.784246 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/kube-rbac-proxy-node/0.log" Jul 28 15:22:14.803929 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.803899 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 15:22:14.821119 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.821096 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/northd/0.log" Jul 28 15:22:14.839260 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.839236 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/nbdb/0.log" Jul 28 15:22:14.857959 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.857937 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/sbdb/0.log" Jul 28 15:22:14.914996 ip-10-0-131-88 kubenswrapper[2586]: I0728 15:22:14.914964 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-crhnz_7167f2b7-efb4-42ac-a941-847c75bbeb89/ovnkube-controller/0.log"