Jul 06 12:46:39.786440 ip-10-0-136-21 systemd[1]: Starting Kubernetes Kubelet... Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.239108 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244523 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244536 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244543 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244547 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244550 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244553 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 12:46:40.273593 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244556 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244559 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244561 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244564 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244567 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244569 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244572 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244575 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244577 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244580 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244582 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244586 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244589 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244592 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244596 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244600 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244603 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244606 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244609 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 12:46:40.275752 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244611 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244614 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244618 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244621 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244624 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244627 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244630 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244633 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244635 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244638 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244641 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244644 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244647 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244649 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244652 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244655 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244658 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244661 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244663 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244666 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 12:46:40.293810 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244669 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244672 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244675 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244678 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244682 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244685 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244688 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244691 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244694 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244697 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244700 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244702 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244705 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244707 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244710 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244713 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244716 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244718 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244721 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244724 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 12:46:40.324302 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244726 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 12:46:40.315214 ip-10-0-136-21 systemd[1]: Started Kubernetes Kubelet. Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244729 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244731 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244734 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244737 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244739 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244742 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244744 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244747 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244750 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244755 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244758 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244761 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244763 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244766 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244768 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244771 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244773 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244776 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244779 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 12:46:40.341099 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244781 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244784 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244787 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244789 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244792 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244794 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244797 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244800 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244802 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244805 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244808 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244810 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244813 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.244816 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245543 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245550 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245553 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245556 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245559 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245561 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 12:46:40.357001 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245564 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245568 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245571 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245574 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245577 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245580 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245582 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245585 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245588 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245591 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245594 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245596 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245599 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245601 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245604 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245607 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245610 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245613 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245615 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245618 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 12:46:40.358378 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245621 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245623 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245626 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245628 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245631 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245634 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245636 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245639 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245642 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245644 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245647 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245650 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245652 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245655 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245659 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245662 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245666 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245668 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245671 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 12:46:40.376268 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245673 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245676 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245679 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245683 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245685 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245688 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245691 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245694 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245697 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245700 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245703 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245706 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245708 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245711 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245713 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245716 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245719 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245721 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245724 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245726 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 12:46:40.392725 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245729 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245734 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245737 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245739 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245742 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245744 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245747 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245749 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245752 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245755 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245759 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245763 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245767 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245770 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245774 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245777 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245780 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245783 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245785 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 12:46:40.409682 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245788 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245791 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245794 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245798 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245801 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245803 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245806 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245808 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245810 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245813 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245816 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245818 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245821 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245824 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.245828 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246205 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246210 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246213 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246216 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246219 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 12:46:40.563875 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246222 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246226 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246230 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246233 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246236 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246239 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246242 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246245 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246247 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246250 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246253 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246255 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246260 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246263 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246266 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246268 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246271 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246273 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246278 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 12:46:40.564706 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246282 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246285 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246288 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246290 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246293 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246296 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246298 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246302 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246305 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246308 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246310 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246313 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246316 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246318 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246321 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246323 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246326 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246329 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246331 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246334 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 12:46:40.566770 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246336 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246339 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246341 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246344 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246347 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246350 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246353 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246356 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246358 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246361 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246365 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246368 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246370 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246373 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246375 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246378 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246381 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246383 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246386 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246389 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 12:46:40.567418 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246392 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246394 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246397 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246399 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246402 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246405 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246409 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246412 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246414 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246417 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246420 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246423 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246425 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246427 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246430 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246432 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246434 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246439 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246441 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246444 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 12:46:40.568117 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246446 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246449 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246451 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246454 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246456 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246459 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246461 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246464 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246466 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246469 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246471 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246475 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246477 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246480 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.246482 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246552 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246559 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246565 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246570 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246574 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246577 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 12:46:40.692559 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246581 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246586 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246589 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246593 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246596 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246599 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246602 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246605 2559 flags.go:64] FLAG: --cgroup-root="" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246610 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246613 2559 flags.go:64] FLAG: --client-ca-file="" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246616 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246619 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246624 2559 flags.go:64] FLAG: --cluster-domain="" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246627 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246630 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246633 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246636 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246639 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246642 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246645 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246649 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246652 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246656 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246659 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 12:46:40.693266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246662 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246665 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246669 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246672 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246675 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246677 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246680 2559 flags.go:64] FLAG: --enable-server="true" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246684 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246688 2559 flags.go:64] FLAG: --event-burst="100" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246691 2559 flags.go:64] FLAG: --event-qps="50" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246694 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246701 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246705 2559 flags.go:64] FLAG: --eviction-hard="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246708 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246711 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246714 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246718 2559 flags.go:64] FLAG: --eviction-soft="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246721 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246724 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246727 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246730 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246732 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246735 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246738 2559 flags.go:64] FLAG: --feature-gates="" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246742 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 12:46:40.693983 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246745 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246748 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246751 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246754 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246757 2559 flags.go:64] FLAG: --help="false" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246761 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246764 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246768 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246770 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246774 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246777 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246780 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246782 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246785 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246788 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246791 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246794 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246797 2559 flags.go:64] FLAG: --kube-reserved="" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246799 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246803 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246806 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246809 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246811 2559 flags.go:64] FLAG: --lock-file="" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246815 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 12:46:40.694752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246818 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246821 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246826 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246829 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246832 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246835 2559 flags.go:64] FLAG: --logging-format="text" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246838 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246841 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246843 2559 flags.go:64] FLAG: --manifest-url="" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246846 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246851 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246854 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246858 2559 flags.go:64] FLAG: --max-pods="110" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246862 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246865 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246868 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246871 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246874 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246877 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246880 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246886 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246889 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246892 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 12:46:40.695503 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246895 2559 flags.go:64] FLAG: --pod-cidr="" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246898 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246903 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246906 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246909 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246913 2559 flags.go:64] FLAG: --port="10250" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246916 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246918 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0694e881278cf9fca" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246922 2559 flags.go:64] FLAG: --qos-reserved="" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246926 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246929 2559 flags.go:64] FLAG: --register-node="true" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246932 2559 flags.go:64] FLAG: --register-with-taints="" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246935 2559 flags.go:64] FLAG: --registry-burst="10" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246938 2559 flags.go:64] FLAG: --registry-qps="5" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246941 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246943 2559 flags.go:64] FLAG: --reserved-memory="" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246947 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246950 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246953 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246956 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246959 2559 flags.go:64] FLAG: --runonce="false" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246962 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246966 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246969 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246972 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 12:46:40.696107 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246975 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246978 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246981 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246984 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246987 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246990 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246993 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246996 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.246999 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247002 2559 flags.go:64] FLAG: --system-cgroups="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247005 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247010 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247012 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247016 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247020 2559 flags.go:64] FLAG: --tls-min-version="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247023 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247027 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247030 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247033 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247036 2559 flags.go:64] FLAG: --v="2" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247039 2559 flags.go:64] FLAG: --version="false" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247048 2559 flags.go:64] FLAG: --vmodule="" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247052 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.247055 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247186 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 12:46:40.696811 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247191 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247194 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247197 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247200 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247204 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247207 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247209 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247212 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247214 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247217 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247219 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247222 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247224 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247227 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247230 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247232 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247235 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247237 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247241 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247244 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 12:46:40.697528 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247246 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247249 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247252 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247258 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247261 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247264 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247267 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247269 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247272 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247275 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247277 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247280 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247282 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247285 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247288 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247290 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247295 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247297 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247300 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247302 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 12:46:40.698088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247305 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247309 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247313 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247316 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247319 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247322 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247324 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247327 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247330 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247333 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247336 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247339 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247341 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247344 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247347 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247351 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247354 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247357 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247359 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 12:46:40.698640 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247362 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247364 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247367 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247369 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247372 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247375 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247377 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247380 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247383 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247387 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247389 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247392 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247395 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247397 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247399 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247402 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247404 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247407 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247409 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247412 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 12:46:40.699199 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247414 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247417 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247419 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247422 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247426 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247429 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247432 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247435 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247439 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247441 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247444 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247446 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247449 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247452 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247454 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247457 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247459 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247462 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 12:46:40.699740 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.247464 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.248259 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.255001 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.255016 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.255036 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.255045 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255101 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255106 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255109 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255112 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255116 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255119 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255121 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255124 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255127 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 12:46:40.700266 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255129 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255132 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255135 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255137 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255140 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255144 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255147 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255150 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255153 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255156 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255178 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255183 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255188 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255192 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255197 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255201 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255205 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255209 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255213 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 12:46:40.700686 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255217 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255222 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255226 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255231 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255235 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255238 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255241 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255245 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255248 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255253 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255257 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255261 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255265 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255269 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255273 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255276 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255280 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255284 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255287 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255292 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 12:46:40.701294 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255295 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255299 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255303 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255308 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255311 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255315 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255319 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255325 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255329 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255333 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255338 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255342 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255346 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255349 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255354 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255357 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255361 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255366 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255369 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255373 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255376 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 12:46:40.701970 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255380 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255384 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255387 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255391 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255395 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255398 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255402 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255408 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255412 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255416 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255420 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255423 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255427 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255431 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255434 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255438 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255444 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255449 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255455 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 12:46:40.702667 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255458 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255462 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255467 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255472 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255475 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255480 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255483 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255488 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255491 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255495 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255498 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.255505 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255670 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255678 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255681 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 12:46:40.705152 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255685 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255689 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255693 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255698 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255702 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255707 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255711 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255714 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255718 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255722 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255725 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255729 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255733 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255737 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255741 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255744 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255748 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255753 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255757 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255760 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 12:46:40.705777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255764 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255768 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255772 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255775 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255779 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255783 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255787 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255791 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255797 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255801 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255805 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255809 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255813 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255817 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255820 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255825 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255828 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255833 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255838 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 12:46:40.841939 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255841 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255845 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255848 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255852 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255857 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255862 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255866 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255869 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255873 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255876 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255882 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255885 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255889 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255893 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255897 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255900 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255903 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255906 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255910 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 12:46:40.842744 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255914 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255918 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255922 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255925 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255929 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255932 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255936 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255940 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255946 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255950 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255954 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255959 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255963 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255966 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255970 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255974 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255977 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255979 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255982 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255984 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 12:46:40.843571 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255987 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255989 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255992 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255995 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.255998 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256001 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256003 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256006 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256008 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256011 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256013 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256016 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256019 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256022 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256024 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256027 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256029 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 12:46:40.844417 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:40.256032 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.256037 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.256158 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.258665 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.260158 2559 server.go:1019] "Starting client certificate rotation" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.260274 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.260305 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.262914 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.264052 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.279746 2559 log.go:25] "Validated CRI v1 runtime API" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.282874 2559 log.go:25] "Validated CRI v1 image API" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.283661 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 12:46:40.845613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.286209 2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 92432af1-2b27-4d0d-956a-5bb54d0bd33e:/dev/nvme0n1p4 9e373df9-b085-4bec-8bc2-940f8da657a9:/dev/nvme0n1p3] Jul 06 12:46:40.846012 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.286224 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.290964 2559 manager.go:216] Machine: {Timestamp:2026-07-06 12:46:40.289636703 +0000 UTC m=+0.371077721 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092336 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ae2d93b5d3357c32bed6e71e78994 SystemUUID:ec2ae2d9-3b5d-3357-c32b-ed6e71e78994 BootID:9e07cf76-10af-4540-8ad7-09474da8998f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:35:e8:35:8b:8b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:35:e8:35:8b:8b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:f0:fa:9b:f3:ad Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.291077 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.291212 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.296041 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.296065 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-21.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.297260 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.297270 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 12:46:40.846839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.297277 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.297890 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.299005 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302065 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302226 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302385 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302403 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302417 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302427 2559 kubelet.go:398] "Adding apiserver pod source" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302436 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302870 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.302905 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.304402 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.305665 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.307686 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.307719 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309154 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309185 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309193 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309201 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309210 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309217 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309226 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309231 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309239 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309244 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309253 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.309259 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.310619 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.310628 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.314263 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-21.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.314377 2559 server.go:1268] "Started kubelet" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.314499 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.314521 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.314549 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.314509 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.314671 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.319697 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.322010 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 06 12:46:40.847263 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.323620 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.326010 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419c9c1511b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.314077467 +0000 UTC m=+0.395518486,LastTimestamp:2026-07-06 12:46:40.314077467 +0000 UTC m=+0.395518486,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327074 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327082 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.327193 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327741 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327762 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327807 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327854 2559 factory.go:55] Registering systemd factory Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327869 2559 factory.go:223] Registration of the systemd container factory successfully Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327949 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.327959 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.328060 2559 factory.go:153] Registering CRI-O factory Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.328072 2559 factory.go:223] Registration of the crio container factory successfully Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.328122 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.328150 2559 factory.go:103] Registering Raw factory Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.328182 2559 manager.go:1196] Started watching for new ooms in manager Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.328648 2559 manager.go:349] Starting recovery of all containers Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.336560 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.336599 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-21.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.338517 2559 manager.go:354] Recovery completed Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.343134 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346129 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346160 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346192 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346656 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346664 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346669 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.346677 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.347423 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.347446 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.347456 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.347869 2559 policy_none.go:49] "None policy: Start" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.347881 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 12:46:40.848489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.347891 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 12:46:40.849709 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.348697 2559 policy_none.go:47] "Start" Jul 06 12:46:40.849709 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.348655 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419cbaa8ed2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.34614037 +0000 UTC m=+0.427581387,LastTimestamp:2026-07-06 12:46:40.34614037 +0000 UTC m=+0.427581387,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.849709 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.361543 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419cbab3386 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.346182534 +0000 UTC m=+0.427623558,LastTimestamp:2026-07-06 12:46:40.346182534 +0000 UTC m=+0.427623558,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.849709 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.376283 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419cbab6f8f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.346197903 +0000 UTC m=+0.427638922,LastTimestamp:2026-07-06 12:46:40.346197903 +0000 UTC m=+0.427638922,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.849709 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.388950 2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-21.ec2.internal.18bfb419cbaa8ed2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419cbaa8ed2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.34614037 +0000 UTC m=+0.427581387,LastTimestamp:2026-07-06 12:46:40.347433573 +0000 UTC m=+0.428874591,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.403202 2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-21.ec2.internal.18bfb419cbab3386\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419cbab3386 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.346182534 +0000 UTC m=+0.427623558,LastTimestamp:2026-07-06 12:46:40.347450572 +0000 UTC m=+0.428891589,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.407846 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.409085 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.409122 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.409145 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.409201 2559 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.419715 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.420217 2559 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-21.ec2.internal.18bfb419cbab6f8f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419cbab6f8f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-21.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.346197903 +0000 UTC m=+0.427638922,LastTimestamp:2026-07-06 12:46:40.347459994 +0000 UTC m=+0.428901013,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.427671 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.439720 2559 manager.go:337] "Starting Device Plugin manager" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.439798 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.439812 2559 server.go:87] "Starting device plugin registration server" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.440082 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.440094 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.440191 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.440267 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.440276 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.440820 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 12:46:40.850116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.440869 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.444204 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-21.ec2.internal.18bfb419d16067de default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-21.ec2.internal,UID:ip-10-0-136-21.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-136-21.ec2.internal,},FirstTimestamp:2026-07-06 12:46:40.44194403 +0000 UTC m=+0.523385037,LastTimestamp:2026-07-06 12:46:40.44194403 +0000 UTC m=+0.523385037,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-21.ec2.internal,}" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.472339 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9sgfr" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.482732 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9sgfr" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.510111 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"] Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.510201 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.511843 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.511877 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.511889 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513041 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513153 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513191 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513717 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513736 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513743 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513754 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513760 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.513763 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.514876 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.514904 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.515577 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.515597 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.515606 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.528955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/250d02a473ed274be5fbcb524906c9f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"250d02a473ed274be5fbcb524906c9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.850913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.528977 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99f360b5a4f3a206c7dffeae18c1b293-config\") pod \"kube-apiserver-proxy-ip-10-0-136-21.ec2.internal\" (UID: \"99f360b5a4f3a206c7dffeae18c1b293\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.528994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/250d02a473ed274be5fbcb524906c9f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"250d02a473ed274be5fbcb524906c9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.540142 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.540989 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientMemory" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.541015 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.541025 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeHasSufficientPID" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.541062 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.543014 2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-21.ec2.internal\" not found" node="ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.544819 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-21.ec2.internal\" not found" node="ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.549209 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-21.ec2.internal\" not found" node="ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.550202 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.550219 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-21.ec2.internal\": node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.567700 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.631401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/250d02a473ed274be5fbcb524906c9f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"250d02a473ed274be5fbcb524906c9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.631577 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/250d02a473ed274be5fbcb524906c9f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"250d02a473ed274be5fbcb524906c9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.631596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99f360b5a4f3a206c7dffeae18c1b293-config\") pod \"kube-apiserver-proxy-ip-10-0-136-21.ec2.internal\" (UID: \"99f360b5a4f3a206c7dffeae18c1b293\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.631492 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/250d02a473ed274be5fbcb524906c9f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"250d02a473ed274be5fbcb524906c9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.631632 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99f360b5a4f3a206c7dffeae18c1b293-config\") pod \"kube-apiserver-proxy-ip-10-0-136-21.ec2.internal\" (UID: \"99f360b5a4f3a206c7dffeae18c1b293\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.631672 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/250d02a473ed274be5fbcb524906c9f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal\" (UID: \"250d02a473ed274be5fbcb524906c9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.668206 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.772884 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:40.852156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.850527 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:41.080514 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:40.992443 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" Jul 06 12:46:41.080514 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:40.992444 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:41.094011 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.093542 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:41.193704 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.147888 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 12:46:41.194912 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.194520 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-21.ec2.internal\" not found" Jul 06 12:46:41.310393 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.258955 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 12:46:41.310393 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.259117 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 12:46:41.310393 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.282858 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 12:46:41.310393 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.285416 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 12:46:41.310393 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.303262 2559 apiserver.go:52] "Watching apiserver" Jul 06 12:46:41.314076 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.313367 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 12:46:41.314076 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.313735 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-jkfvl","kube-system/konnectivity-agent-z4j4h","openshift-dns/node-resolver-4sns4","openshift-image-registry/node-ca-6vxs7","openshift-multus/multus-9fkdq","openshift-multus/multus-additional-cni-plugins-ngzm5","openshift-multus/network-metrics-daemon-bflvj","openshift-ovn-kubernetes/ovnkube-node-tgmrj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh","openshift-cluster-node-tuning-operator/tuned-7rpwr","openshift-network-diagnostics/network-check-target-6xs84","openshift-network-operator/iptables-alerter-6z56m"] Jul 06 12:46:41.315903 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.315362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.315903 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.315475 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:41.319253 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.317801 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.319253 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.319007 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321118 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321248 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321329 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t8rcr\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321443 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321452 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321695 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.321748 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qztz5\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.322393 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.323351 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.324015 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.324389 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.324438 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.324550 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.325541 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.325757 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.325957 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.325998 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.326711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.326654 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327024 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327054 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327061 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327215 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327246 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l86zk\"" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327283 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327284 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zngrq\"" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327217 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 12:46:41.327693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327501 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sphd2\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.327992 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.329364 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.329490 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.329565 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.329620 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.329662 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.329675 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.330302 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.330733 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xnllj\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.330941 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.331410 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4z46g\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332052 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332069 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332141 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332195 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332525 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8cszw\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332550 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.332789 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.333159 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.333297 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.333527 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d9h5k\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.333759 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336070 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-device-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336098 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-sys\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.443074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b1b9765-789d-4fc8-bb16-10601e312f90-hosts-file\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336137 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-os-release\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-var-lib-kubelet\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336193 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1cb7806-30f8-40b1-9cff-0dc679136c5a-host\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336207 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-conf-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336220 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336269 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-ovn\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-run-ovn-kubernetes\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysctl-conf\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336349 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-host\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336369 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/850b8299-9c34-4e86-9d54-95161af94888-konnectivity-ca\") pod \"konnectivity-agent-z4j4h\" (UID: \"850b8299-9c34-4e86-9d54-95161af94888\") " pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b1b9765-789d-4fc8-bb16-10601e312f90-tmp-dir\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336416 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rptz6\" (UniqueName: \"kubernetes.io/projected/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-kube-api-access-rptz6\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336434 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1cb7806-30f8-40b1-9cff-0dc679136c5a-serviceca\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.444768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-node-log\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336460 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4d52\" (UniqueName: \"kubernetes.io/projected/41e2b5df-fdbb-491d-962e-829c3f6f3405-kube-api-access-t4d52\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336474 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336513 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbd2p\" (UniqueName: \"kubernetes.io/projected/3b1b9765-789d-4fc8-bb16-10601e312f90-kube-api-access-sbd2p\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336530 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-cni-multus\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336545 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysctl-d\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336557 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-run\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/275f73b3-0289-47cb-b011-e999f05da7f3-tmp\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8h5w\" (UniqueName: \"kubernetes.io/projected/a1cb7806-30f8-40b1-9cff-0dc679136c5a-kube-api-access-n8h5w\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336633 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aceb5505-409d-4237-9536-6c50269aac4f-multus-daemon-config\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-etc-kubernetes\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336702 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336737 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-kubelet\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/956e4c01-2aab-4200-8d08-b03df4427e55-dbus\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.445731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336781 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-cni-bin\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-slash\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336847 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-systemd\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336871 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-modprobe-d\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336892 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336926 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovnkube-config\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-registration-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336968 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-kubernetes\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.336990 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovn-node-metrics-cert\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337011 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-cnibin\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48dhw\" (UniqueName: \"kubernetes.io/projected/9b3622d5-e204-41c2-ba3b-a76597588b25-kube-api-access-48dhw\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337053 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-system-cni-dir\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.446986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337099 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z9pf\" (UniqueName: \"kubernetes.io/projected/9ff1fa77-47ee-440b-9656-63ff8211a040-kube-api-access-7z9pf\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-cni-bin\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/275f73b3-0289-47cb-b011-e999f05da7f3-etc-tuned\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337178 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-multus-certs\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337207 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-lib-modules\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/850b8299-9c34-4e86-9d54-95161af94888-agent-certs\") pod \"konnectivity-agent-z4j4h\" (UID: \"850b8299-9c34-4e86-9d54-95161af94888\") " pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337264 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-os-release\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aceb5505-409d-4237-9536-6c50269aac4f-cni-binary-copy\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rvfq\" (UniqueName: \"kubernetes.io/projected/aceb5505-409d-4237-9536-6c50269aac4f-kube-api-access-6rvfq\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-run-netns\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337360 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-var-lib-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysconfig\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-hostroot\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.448063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337423 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-systemd-units\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337439 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337454 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovnkube-script-lib\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337469 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfdks\" (UniqueName: \"kubernetes.io/projected/275f73b3-0289-47cb-b011-e999f05da7f3-kube-api-access-kfdks\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337484 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-socket-dir-parent\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337945 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-sys-fs\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-systemd\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.337995 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/956e4c01-2aab-4200-8d08-b03df4427e55-kubelet-config\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338011 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-system-cni-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338029 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-cni-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338051 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-k8s-cni-cncf-io\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-netns\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.449781 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-cnibin\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338095 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-log-socket\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-cni-netd\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338139 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-socket-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338153 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-kubelet\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-etc-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-env-overrides\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.338231 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.349417 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.353227 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.353291 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.353605 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal"] Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.362121 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.362145 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal"] Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.428947 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438330 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.450575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438356 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-etc-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-env-overrides\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-etc-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438480 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-device-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-sys\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438535 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b1b9765-789d-4fc8-bb16-10601e312f90-hosts-file\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-os-release\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438565 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-var-lib-kubelet\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438593 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-device-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438597 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-sys\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438601 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f004d9ed-be2a-4208-81a3-62fb8187f3c3-host-slash\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438629 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-var-lib-kubelet\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438645 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b1b9765-789d-4fc8-bb16-10601e312f90-hosts-file\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.451779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1cb7806-30f8-40b1-9cff-0dc679136c5a-host\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-conf-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438679 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1cb7806-30f8-40b1-9cff-0dc679136c5a-host\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438697 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438709 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-conf-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-ovn\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438733 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-ovn\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438717 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-os-release\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-run-ovn-kubernetes\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438762 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysctl-conf\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438779 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-host\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.438785 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438792 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-run-ovn-kubernetes\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438798 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/850b8299-9c34-4e86-9d54-95161af94888-konnectivity-ca\") pod \"konnectivity-agent-z4j4h\" (UID: \"850b8299-9c34-4e86-9d54-95161af94888\") " pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b1b9765-789d-4fc8-bb16-10601e312f90-tmp-dir\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.452602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438853 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-host\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rptz6\" (UniqueName: \"kubernetes.io/projected/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-kube-api-access-rptz6\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.438904 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:41.93886845 +0000 UTC m=+2.020309489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438920 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysctl-conf\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1cb7806-30f8-40b1-9cff-0dc679136c5a-serviceca\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-node-log\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438974 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4d52\" (UniqueName: \"kubernetes.io/projected/41e2b5df-fdbb-491d-962e-829c3f6f3405-kube-api-access-t4d52\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.438997 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439052 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-node-log\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439044 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbd2p\" (UniqueName: \"kubernetes.io/projected/3b1b9765-789d-4fc8-bb16-10601e312f90-kube-api-access-sbd2p\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-cni-multus\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439178 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysctl-d\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439195 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-cni-multus\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-run\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.453423 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439198 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-etc-selinux\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439228 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/275f73b3-0289-47cb-b011-e999f05da7f3-tmp\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439250 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8h5w\" (UniqueName: \"kubernetes.io/projected/a1cb7806-30f8-40b1-9cff-0dc679136c5a-kube-api-access-n8h5w\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aceb5505-409d-4237-9536-6c50269aac4f-multus-daemon-config\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439311 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-etc-kubernetes\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-run\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439332 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysctl-d\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439409 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-etc-kubernetes\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439335 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439515 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-kubelet\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439537 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/956e4c01-2aab-4200-8d08-b03df4427e55-dbus\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439558 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439565 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-kubelet\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-cni-bin\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439601 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-cni-bin\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.454254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439603 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-slash\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-systemd\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-modprobe-d\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439636 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/956e4c01-2aab-4200-8d08-b03df4427e55-dbus\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439668 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-slash\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439670 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-systemd\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.439670 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439714 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439729 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-modprobe-d\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439685 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-run-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.439781 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:41.939765629 +0000 UTC m=+2.021206651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439813 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovnkube-config\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439838 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-registration-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439860 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-kubernetes\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.455063 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439885 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-registration-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f004d9ed-be2a-4208-81a3-62fb8187f3c3-iptables-alerter-script\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovn-node-metrics-cert\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439930 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-kubernetes\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-cnibin\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.439970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48dhw\" (UniqueName: \"kubernetes.io/projected/9b3622d5-e204-41c2-ba3b-a76597588b25-kube-api-access-48dhw\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440027 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-cnibin\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-system-cni-dir\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z9pf\" (UniqueName: \"kubernetes.io/projected/9ff1fa77-47ee-440b-9656-63ff8211a040-kube-api-access-7z9pf\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440145 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-cni-bin\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/275f73b3-0289-47cb-b011-e999f05da7f3-etc-tuned\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-multus-certs\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440228 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-lib-modules\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.455911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440067 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-system-cni-dir\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440257 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440303 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9sh2\" (UniqueName: \"kubernetes.io/projected/f004d9ed-be2a-4208-81a3-62fb8187f3c3-kube-api-access-l9sh2\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440320 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-multus-certs\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440332 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/850b8299-9c34-4e86-9d54-95161af94888-agent-certs\") pod \"konnectivity-agent-z4j4h\" (UID: \"850b8299-9c34-4e86-9d54-95161af94888\") " pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-os-release\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440379 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aceb5505-409d-4237-9536-6c50269aac4f-cni-binary-copy\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440415 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rvfq\" (UniqueName: \"kubernetes.io/projected/aceb5505-409d-4237-9536-6c50269aac4f-kube-api-access-6rvfq\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440431 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-run-netns\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440453 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-var-lib-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440467 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysconfig\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440484 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-hostroot\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-systemd-units\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.456707 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440548 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovnkube-script-lib\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440569 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfdks\" (UniqueName: \"kubernetes.io/projected/275f73b3-0289-47cb-b011-e999f05da7f3-kube-api-access-kfdks\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-socket-dir-parent\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440623 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-sys-fs\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-systemd\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440653 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/956e4c01-2aab-4200-8d08-b03df4427e55-kubelet-config\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440668 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-system-cni-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440684 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-cni-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-k8s-cni-cncf-io\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440717 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-netns\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440736 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-cnibin\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440747 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-hostroot\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-log-socket\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.457932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-cni-netd\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440793 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-socket-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440811 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-kubelet\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440818 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-lib-modules\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440828 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-os-release\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440864 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-cni-bin\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440946 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-k8s-cni-cncf-io\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440969 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-system-cni-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440969 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/956e4c01-2aab-4200-8d08-b03df4427e55-kubelet-config\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.440997 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-var-lib-openvswitch\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441019 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-sysconfig\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441038 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441072 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-systemd-units\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441071 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-run-netns\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441107 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-cnibin\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.458731 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441108 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-cni-dir\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441214 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-cni-netd\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441303 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-sys-fs\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441354 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-log-socket\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441359 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41e2b5df-fdbb-491d-962e-829c3f6f3405-host-run-netns\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441406 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/275f73b3-0289-47cb-b011-e999f05da7f3-etc-systemd\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441445 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-host-var-lib-kubelet\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441484 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aceb5505-409d-4237-9536-6c50269aac4f-multus-socket-dir-parent\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.441786 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ff1fa77-47ee-440b-9656-63ff8211a040-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.443249 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-socket-dir\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445265 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-env-overrides\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445395 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovnkube-script-lib\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445448 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovnkube-config\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445457 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.459616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445474 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445490 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ff1fa77-47ee-440b-9656-63ff8211a040-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.445706 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.446466 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1cb7806-30f8-40b1-9cff-0dc679136c5a-serviceca\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.446491 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/850b8299-9c34-4e86-9d54-95161af94888-konnectivity-ca\") pod \"konnectivity-agent-z4j4h\" (UID: \"850b8299-9c34-4e86-9d54-95161af94888\") " pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.446781 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b1b9765-789d-4fc8-bb16-10601e312f90-tmp-dir\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.446853 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aceb5505-409d-4237-9536-6c50269aac4f-cni-binary-copy\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.446921 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aceb5505-409d-4237-9536-6c50269aac4f-multus-daemon-config\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.448945 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/275f73b3-0289-47cb-b011-e999f05da7f3-tmp\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.448992 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/275f73b3-0289-47cb-b011-e999f05da7f3-etc-tuned\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.449089 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/41e2b5df-fdbb-491d-962e-829c3f6f3405-ovn-node-metrics-cert\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.460643 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.449286 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/850b8299-9c34-4e86-9d54-95161af94888-agent-certs\") pod \"konnectivity-agent-z4j4h\" (UID: \"850b8299-9c34-4e86-9d54-95161af94888\") " pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:41.461797 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.461068 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8h5w\" (UniqueName: \"kubernetes.io/projected/a1cb7806-30f8-40b1-9cff-0dc679136c5a-kube-api-access-n8h5w\") pod \"node-ca-6vxs7\" (UID: \"a1cb7806-30f8-40b1-9cff-0dc679136c5a\") " pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.461851 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z9pf\" (UniqueName: \"kubernetes.io/projected/9ff1fa77-47ee-440b-9656-63ff8211a040-kube-api-access-7z9pf\") pod \"multus-additional-cni-plugins-ngzm5\" (UID: \"9ff1fa77-47ee-440b-9656-63ff8211a040\") " pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.461857 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rptz6\" (UniqueName: \"kubernetes.io/projected/d958aea6-0cd4-4c1c-96eb-dc7a9149bab7-kube-api-access-rptz6\") pod \"aws-ebs-csi-driver-node-dtpgh\" (UID: \"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.463421 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfdks\" (UniqueName: \"kubernetes.io/projected/275f73b3-0289-47cb-b011-e999f05da7f3-kube-api-access-kfdks\") pod \"tuned-7rpwr\" (UID: \"275f73b3-0289-47cb-b011-e999f05da7f3\") " pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.463423 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbd2p\" (UniqueName: \"kubernetes.io/projected/3b1b9765-789d-4fc8-bb16-10601e312f90-kube-api-access-sbd2p\") pod \"node-resolver-4sns4\" (UID: \"3b1b9765-789d-4fc8-bb16-10601e312f90\") " pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.463587 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48dhw\" (UniqueName: \"kubernetes.io/projected/9b3622d5-e204-41c2-ba3b-a76597588b25-kube-api-access-48dhw\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.463732 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rvfq\" (UniqueName: \"kubernetes.io/projected/aceb5505-409d-4237-9536-6c50269aac4f-kube-api-access-6rvfq\") pod \"multus-9fkdq\" (UID: \"aceb5505-409d-4237-9536-6c50269aac4f\") " pod="openshift-multus/multus-9fkdq" Jul 06 12:46:41.466885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.463840 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4d52\" (UniqueName: \"kubernetes.io/projected/41e2b5df-fdbb-491d-962e-829c3f6f3405-kube-api-access-t4d52\") pod \"ovnkube-node-tgmrj\" (UID: \"41e2b5df-fdbb-491d-962e-829c3f6f3405\") " pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.484429 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2ww9q" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.487508 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 12:41:41 +0000 UTC" deadline="2028-04-14 09:18:33.653203026 +0000 UTC" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.487548 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15548h31m52.165658189s" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.494791 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2ww9q" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.541498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f004d9ed-be2a-4208-81a3-62fb8187f3c3-iptables-alerter-script\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.541524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.541544 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9sh2\" (UniqueName: \"kubernetes.io/projected/f004d9ed-be2a-4208-81a3-62fb8187f3c3-kube-api-access-l9sh2\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.541578 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f004d9ed-be2a-4208-81a3-62fb8187f3c3-host-slash\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.541634 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f004d9ed-be2a-4208-81a3-62fb8187f3c3-host-slash\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.542082 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f004d9ed-be2a-4208-81a3-62fb8187f3c3-iptables-alerter-script\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.547582 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.547597 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.547606 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.547660 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:46:42.047647865 +0000 UTC m=+2.129088869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:41.574156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.549548 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9sh2\" (UniqueName: \"kubernetes.io/projected/f004d9ed-be2a-4208-81a3-62fb8187f3c3-kube-api-access-l9sh2\") pod \"iptables-alerter-6z56m\" (UID: \"f004d9ed-be2a-4208-81a3-62fb8187f3c3\") " pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:41.950885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.944516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:41.950885 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.944558 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:41.950885 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.944661 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:41.950885 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.944676 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:41.950885 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.944717 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:42.944703162 +0000 UTC m=+3.026144172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:41.950885 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:41.944736 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:42.944722821 +0000 UTC m=+3.026163831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:42.084186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:41.998294 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 12:46:42.217978 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.145921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:42.217978 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.146074 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:46:42.217978 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.146098 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:46:42.217978 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.146109 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:42.217978 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.146164 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:46:43.146149865 +0000 UTC m=+3.227590871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:42.355612 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.285191 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:46:42.355612 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.308382 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4sns4" Jul 06 12:46:42.355612 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.314626 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6vxs7" Jul 06 12:46:42.355612 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.338536 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9fkdq" Jul 06 12:46:42.355612 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.344228 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" Jul 06 12:46:42.500496 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.497534 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 12:41:42 +0000 UTC" deadline="2028-03-26 19:45:49.808484108 +0000 UTC" Jul 06 12:46:42.500496 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.497564 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15102h59m7.310924582s" Jul 06 12:46:42.618604 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.609579 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:46:42.625741 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.619291 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" Jul 06 12:46:42.662498 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.662475 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" Jul 06 12:46:42.670542 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.670521 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6z56m" Jul 06 12:46:42.830991 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.830958 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f360b5a4f3a206c7dffeae18c1b293.slice/crio-4550665e008a3f61a4c6b90d72d86fa6f2817d8f061c83d24570dbac652c4880 WatchSource:0}: Error finding container 4550665e008a3f61a4c6b90d72d86fa6f2817d8f061c83d24570dbac652c4880: Status 404 returned error can't find the container with id 4550665e008a3f61a4c6b90d72d86fa6f2817d8f061c83d24570dbac652c4880 Jul 06 12:46:42.833608 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.833581 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850b8299_9c34_4e86_9d54_95161af94888.slice/crio-a33827e1a3414bc26fac05f132bb501b4ad9a5e7491293c668dcc0a4a2187e85 WatchSource:0}: Error finding container a33827e1a3414bc26fac05f132bb501b4ad9a5e7491293c668dcc0a4a2187e85: Status 404 returned error can't find the container with id a33827e1a3414bc26fac05f132bb501b4ad9a5e7491293c668dcc0a4a2187e85 Jul 06 12:46:42.839692 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.839670 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 12:46:42.849351 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.849328 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250d02a473ed274be5fbcb524906c9f6.slice/crio-4366e59c35083bb3e2071400ad3efa90d132ef573dc173b02003e0e3236ad8b0 WatchSource:0}: Error finding container 4366e59c35083bb3e2071400ad3efa90d132ef573dc173b02003e0e3236ad8b0: Status 404 returned error can't find the container with id 4366e59c35083bb3e2071400ad3efa90d132ef573dc173b02003e0e3236ad8b0 Jul 06 12:46:42.853540 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.853508 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff1fa77_47ee_440b_9656_63ff8211a040.slice/crio-44ba23b4a9a7ce505784085a8040157aad1dcdf20bc917b0c9865fe9592d9850 WatchSource:0}: Error finding container 44ba23b4a9a7ce505784085a8040157aad1dcdf20bc917b0c9865fe9592d9850: Status 404 returned error can't find the container with id 44ba23b4a9a7ce505784085a8040157aad1dcdf20bc917b0c9865fe9592d9850 Jul 06 12:46:42.857705 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.857671 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaceb5505_409d_4237_9536_6c50269aac4f.slice/crio-28b21fc94f791a060b0557046737b62af52b64931a246b1a7ad670aeadbd61db WatchSource:0}: Error finding container 28b21fc94f791a060b0557046737b62af52b64931a246b1a7ad670aeadbd61db: Status 404 returned error can't find the container with id 28b21fc94f791a060b0557046737b62af52b64931a246b1a7ad670aeadbd61db Jul 06 12:46:42.862511 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.862474 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cb7806_30f8_40b1_9cff_0dc679136c5a.slice/crio-afe558cef6409322eb22e9c040c9f54e140cc75416d5a0ebd5d25ddbce06ada4 WatchSource:0}: Error finding container afe558cef6409322eb22e9c040c9f54e140cc75416d5a0ebd5d25ddbce06ada4: Status 404 returned error can't find the container with id afe558cef6409322eb22e9c040c9f54e140cc75416d5a0ebd5d25ddbce06ada4 Jul 06 12:46:42.865348 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:42.865326 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1b9765_789d_4fc8_bb16_10601e312f90.slice/crio-785bf5bc068f09acc6b846bd03fa4256d290bef5f6f398c6ee34c63efb43d69d WatchSource:0}: Error finding container 785bf5bc068f09acc6b846bd03fa4256d290bef5f6f398c6ee34c63efb43d69d: Status 404 returned error can't find the container with id 785bf5bc068f09acc6b846bd03fa4256d290bef5f6f398c6ee34c63efb43d69d Jul 06 12:46:42.952277 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.952248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:42.952413 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:42.952288 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:42.952461 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.952438 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:42.952528 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.952510 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:44.952476726 +0000 UTC m=+5.033917814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:42.952528 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.952521 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:42.952617 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:42.952561 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:44.952549574 +0000 UTC m=+5.033990596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:43.009768 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:43.009737 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275f73b3_0289_47cb_b011_e999f05da7f3.slice/crio-015300faaca9be33b1c4a14d3a2a445b6f7479caafac4bb59a3d9c18f80f0a10 WatchSource:0}: Error finding container 015300faaca9be33b1c4a14d3a2a445b6f7479caafac4bb59a3d9c18f80f0a10: Status 404 returned error can't find the container with id 015300faaca9be33b1c4a14d3a2a445b6f7479caafac4bb59a3d9c18f80f0a10 Jul 06 12:46:43.031707 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:43.031681 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf004d9ed_be2a_4208_81a3_62fb8187f3c3.slice/crio-e2f600535f35a66b8c7c97eb0b57086c9069cc4800af18b38557b1917f5d9708 WatchSource:0}: Error finding container e2f600535f35a66b8c7c97eb0b57086c9069cc4800af18b38557b1917f5d9708: Status 404 returned error can't find the container with id e2f600535f35a66b8c7c97eb0b57086c9069cc4800af18b38557b1917f5d9708 Jul 06 12:46:43.062314 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:43.062289 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e2b5df_fdbb_491d_962e_829c3f6f3405.slice/crio-451b84f90c6cba51f4e985e8af8478a1a198400102990783e9c4dbfa4877167e WatchSource:0}: Error finding container 451b84f90c6cba51f4e985e8af8478a1a198400102990783e9c4dbfa4877167e: Status 404 returned error can't find the container with id 451b84f90c6cba51f4e985e8af8478a1a198400102990783e9c4dbfa4877167e Jul 06 12:46:43.154911 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.154846 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:43.155028 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.154959 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:46:43.155028 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.154973 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:46:43.155028 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.154982 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:43.155028 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.155027 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:46:45.155013648 +0000 UTC m=+5.236454654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:43.246303 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:46:43.246277 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd958aea6_0cd4_4c1c_96eb_dc7a9149bab7.slice/crio-3c1d66b45f50b4f2aca03374657025c14b8b5e5af5f270e23527988e7fbdcb70 WatchSource:0}: Error finding container 3c1d66b45f50b4f2aca03374657025c14b8b5e5af5f270e23527988e7fbdcb70: Status 404 returned error can't find the container with id 3c1d66b45f50b4f2aca03374657025c14b8b5e5af5f270e23527988e7fbdcb70 Jul 06 12:46:43.409622 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.409530 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:43.409622 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.409559 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:43.410461 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.409699 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:43.410461 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.409760 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:43.410461 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.409778 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:43.410461 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:43.409875 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:43.498517 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.498472 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 12:41:42 +0000 UTC" deadline="2028-03-09 01:32:36.607035625 +0000 UTC" Jul 06 12:46:43.498517 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.498515 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14676h45m53.108525718s" Jul 06 12:46:43.507031 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.506976 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" event={"ID":"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7","Type":"ContainerStarted","Data":"3c1d66b45f50b4f2aca03374657025c14b8b5e5af5f270e23527988e7fbdcb70"} Jul 06 12:46:43.509618 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.509501 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"451b84f90c6cba51f4e985e8af8478a1a198400102990783e9c4dbfa4877167e"} Jul 06 12:46:43.514965 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.514892 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6vxs7" event={"ID":"a1cb7806-30f8-40b1-9cff-0dc679136c5a","Type":"ContainerStarted","Data":"afe558cef6409322eb22e9c040c9f54e140cc75416d5a0ebd5d25ddbce06ada4"} Jul 06 12:46:43.516521 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.516491 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerStarted","Data":"44ba23b4a9a7ce505784085a8040157aad1dcdf20bc917b0c9865fe9592d9850"} Jul 06 12:46:43.522644 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.522538 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" event={"ID":"250d02a473ed274be5fbcb524906c9f6","Type":"ContainerStarted","Data":"4366e59c35083bb3e2071400ad3efa90d132ef573dc173b02003e0e3236ad8b0"} Jul 06 12:46:43.525640 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.525569 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z4j4h" event={"ID":"850b8299-9c34-4e86-9d54-95161af94888","Type":"ContainerStarted","Data":"a33827e1a3414bc26fac05f132bb501b4ad9a5e7491293c668dcc0a4a2187e85"} Jul 06 12:46:43.527136 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.527096 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6z56m" event={"ID":"f004d9ed-be2a-4208-81a3-62fb8187f3c3","Type":"ContainerStarted","Data":"e2f600535f35a66b8c7c97eb0b57086c9069cc4800af18b38557b1917f5d9708"} Jul 06 12:46:43.528802 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.528777 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" event={"ID":"275f73b3-0289-47cb-b011-e999f05da7f3","Type":"ContainerStarted","Data":"015300faaca9be33b1c4a14d3a2a445b6f7479caafac4bb59a3d9c18f80f0a10"} Jul 06 12:46:43.538650 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.538557 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4sns4" event={"ID":"3b1b9765-789d-4fc8-bb16-10601e312f90","Type":"ContainerStarted","Data":"785bf5bc068f09acc6b846bd03fa4256d290bef5f6f398c6ee34c63efb43d69d"} Jul 06 12:46:43.546093 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.546050 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9fkdq" event={"ID":"aceb5505-409d-4237-9536-6c50269aac4f","Type":"ContainerStarted","Data":"28b21fc94f791a060b0557046737b62af52b64931a246b1a7ad670aeadbd61db"} Jul 06 12:46:43.553880 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.553841 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" event={"ID":"99f360b5a4f3a206c7dffeae18c1b293","Type":"ContainerStarted","Data":"4550665e008a3f61a4c6b90d72d86fa6f2817d8f061c83d24570dbac652c4880"} Jul 06 12:46:43.920851 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:43.920814 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 12:46:44.972844 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:44.972803 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:44.973396 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:44.972869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:44.973396 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:44.972990 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:44.973396 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:44.973105 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:48.973072187 +0000 UTC m=+9.054513208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:44.973589 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:44.973002 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:44.973651 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:44.973590 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:48.973571755 +0000 UTC m=+9.055012778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:45.174665 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:45.174626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:45.174889 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.174812 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:46:45.174889 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.174832 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:46:45.174889 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.174845 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:45.175069 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.174915 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:46:49.17489412 +0000 UTC m=+9.256335153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:45.409516 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:45.409429 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:45.409727 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.409569 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:45.410004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:45.409982 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:45.410115 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.410092 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:45.410181 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:45.410162 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:45.410291 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:45.410269 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:47.409785 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:47.409745 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:47.410488 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:47.409885 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:47.410488 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:47.410276 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:47.410488 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:47.410387 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:47.410488 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:47.410436 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:47.410717 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:47.410522 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:49.009099 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:49.008379 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:49.009099 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:49.008446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:49.009099 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.008586 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:49.009099 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.008653 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:57.008632454 +0000 UTC m=+17.090073469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:49.009099 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.009091 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:49.009853 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.009141 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:46:57.009126506 +0000 UTC m=+17.090567516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:49.211196 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:49.211077 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:49.211382 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.211317 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:46:49.211382 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.211346 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:46:49.211382 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.211361 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:49.211556 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.211438 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:46:57.211415196 +0000 UTC m=+17.292856219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:49.409984 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:49.409943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:49.409984 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:49.409968 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:49.410431 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:49.409967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:49.410431 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.410080 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:49.410431 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.410181 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:49.410431 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:49.410278 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:51.410179 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:51.410128 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:51.410179 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:51.410179 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:51.410691 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:51.410270 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:51.410691 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:51.410331 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:51.410691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:51.410384 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:51.410691 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:51.410464 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:53.409838 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:53.409802 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:53.410297 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:53.409802 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:53.410297 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:53.409941 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:53.410297 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:53.409819 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:53.410297 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:53.410013 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:53.410297 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:53.410089 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:55.409590 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:55.409550 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:55.409590 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:55.409588 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:55.410102 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:55.409588 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:55.410102 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:55.409710 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:55.410102 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:55.409833 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:55.410102 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:55.409924 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:57.072873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:57.072844 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:57.073355 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:57.072882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:57.073355 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.072979 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:57.073355 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.073048 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:13.07302758 +0000 UTC m=+33.154468600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:46:57.073355 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.072982 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:57.073355 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.073126 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:13.073111203 +0000 UTC m=+33.154552211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:46:57.274242 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:57.274197 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:57.274440 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.274416 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:46:57.274527 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.274449 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:46:57.274527 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.274464 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:57.274527 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.274525 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:47:13.274507754 +0000 UTC m=+33.355948777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:46:57.409991 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:57.409903 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:57.409991 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:57.409943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:57.409991 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:57.409951 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:57.410259 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.410032 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:57.410259 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.410151 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:57.410353 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:57.410279 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:46:59.410038 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:59.409953 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:46:59.410038 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:59.409981 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:46:59.410497 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:46:59.410082 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:46:59.410497 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:59.410084 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:46:59.410497 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:59.410222 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:46:59.410497 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:46:59.410331 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:01.409743 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:01.409701 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:01.410240 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:01.409809 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:01.410240 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:01.409928 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:01.410240 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:01.409937 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:01.410240 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:01.410027 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:01.410240 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:01.410116 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:03.409633 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:03.409599 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:03.409998 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:03.409701 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:03.409998 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:03.409708 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:03.409998 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:03.409730 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:03.409998 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:03.409813 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:03.409998 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:03.409872 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:04.605623 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.605267 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6vxs7" event={"ID":"a1cb7806-30f8-40b1-9cff-0dc679136c5a","Type":"ContainerStarted","Data":"62a72cba0d5cbe9f66f8d383fb39596724618f312caa72111335940a647c72ff"} Jul 06 12:47:04.606793 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.606769 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ff1fa77-47ee-440b-9656-63ff8211a040" containerID="8f6e2317be3ca3e2468361d150af7c24cd69092cacec1eed088cac276426f50c" exitCode=0 Jul 06 12:47:04.606870 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.606846 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerDied","Data":"8f6e2317be3ca3e2468361d150af7c24cd69092cacec1eed088cac276426f50c"} Jul 06 12:47:04.608301 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.608282 2559 generic.go:356] "Generic (PLEG): container finished" podID="250d02a473ed274be5fbcb524906c9f6" containerID="e7c15ca5612066c0d1014e398f8c0456d0f781b3beef03232d6bc74ade8ce90e" exitCode=0 Jul 06 12:47:04.608401 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.608347 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" event={"ID":"250d02a473ed274be5fbcb524906c9f6","Type":"ContainerDied","Data":"e7c15ca5612066c0d1014e398f8c0456d0f781b3beef03232d6bc74ade8ce90e"} Jul 06 12:47:04.609810 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.609787 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z4j4h" event={"ID":"850b8299-9c34-4e86-9d54-95161af94888","Type":"ContainerStarted","Data":"e8b15e1802ad979348215a13045517acd48e1d414bc68911bced80ae66076422"} Jul 06 12:47:04.611361 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.611338 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" event={"ID":"275f73b3-0289-47cb-b011-e999f05da7f3","Type":"ContainerStarted","Data":"e5735a21f086caaa3bd42aa58c5eb47e8b919bbfc24b4a9567c846ed0407ed74"} Jul 06 12:47:04.612639 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.612620 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4sns4" event={"ID":"3b1b9765-789d-4fc8-bb16-10601e312f90","Type":"ContainerStarted","Data":"2d0d0808adaf6ce10820c8f45cd4bd9b59c9a23f62cd9816de7f174da95e04ce"} Jul 06 12:47:04.614014 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.613992 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9fkdq" event={"ID":"aceb5505-409d-4237-9536-6c50269aac4f","Type":"ContainerStarted","Data":"3f007f2b22b235e2d7837911d15c8d2eb549a9708e610c04d3ff6b2bb9e87302"} Jul 06 12:47:04.615339 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.615320 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" event={"ID":"99f360b5a4f3a206c7dffeae18c1b293","Type":"ContainerStarted","Data":"421d3ede1cc1abe1ef1e1b23d3c689eeeb53e2d648d2b842ceee2b5f62e12466"} Jul 06 12:47:04.616609 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.616590 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" event={"ID":"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7","Type":"ContainerStarted","Data":"860c27f81220acad3ed7b66d27a5b97c288c5be6ac1657a34dd93c7fccb89085"} Jul 06 12:47:04.618816 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.618799 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:47:04.619137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.619119 2559 generic.go:356] "Generic (PLEG): container finished" podID="41e2b5df-fdbb-491d-962e-829c3f6f3405" containerID="55abe16522ca83de366e10f64f7beefec1e528a00e9855faff4aba3e8722a62b" exitCode=1 Jul 06 12:47:04.619213 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.619146 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"7b33f75dc733583aa21042dd12376e75bdfd58d516a6ec8992d5a3a0e74d60a6"} Jul 06 12:47:04.619251 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.619212 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"2b2ee694ce58c35a734378d974af6268db91643a79938afae578255c8d80efaa"} Jul 06 12:47:04.619280 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.619261 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"4b7c5746880de6f1f7ec01ec43e15711deb6261ca581ce6549f9f4b7fa7eae9b"} Jul 06 12:47:04.619315 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.619298 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerDied","Data":"55abe16522ca83de366e10f64f7beefec1e528a00e9855faff4aba3e8722a62b"} Jul 06 12:47:04.619349 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.619328 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"39d8a59bc4eba29fca903a64fb4df570913a05cd891df753edc7a42cedb73cc5"} Jul 06 12:47:04.629796 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.629759 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6vxs7" podStartSLOduration=3.053884479 podStartE2EDuration="23.629748061s" totalImagesPullingTime="20.575863582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:04.629276175 +0000 UTC m=+24.710717201" watchObservedRunningTime="2026-07-06 12:47:04.629748061 +0000 UTC m=+24.711189090" Jul 06 12:47:04.678615 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.678569 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7rpwr" podStartSLOduration=3.249782798 podStartE2EDuration="23.678555372s" totalImagesPullingTime="20.428772574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:04.678359216 +0000 UTC m=+24.759800242" watchObservedRunningTime="2026-07-06 12:47:04.678555372 +0000 UTC m=+24.759996399" Jul 06 12:47:04.678808 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.678790 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z4j4h" podStartSLOduration=3.078425099 podStartE2EDuration="23.678784858s" totalImagesPullingTime="20.600359759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:04.647775995 +0000 UTC m=+24.729217022" watchObservedRunningTime="2026-07-06 12:47:04.678784858 +0000 UTC m=+24.760225943" Jul 06 12:47:04.726381 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.726341 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-21.ec2.internal" podStartSLOduration=23.726324942 podStartE2EDuration="23.726324942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:04.724936535 +0000 UTC m=+24.806377576" watchObservedRunningTime="2026-07-06 12:47:04.726324942 +0000 UTC m=+24.807765966" Jul 06 12:47:04.835903 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.835815 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4sns4" podStartSLOduration=3.243520123 podStartE2EDuration="23.835799168s" totalImagesPullingTime="20.592279045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:04.770356688 +0000 UTC m=+24.851797726" watchObservedRunningTime="2026-07-06 12:47:04.835799168 +0000 UTC m=+24.917240172" Jul 06 12:47:04.836031 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:04.835948 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9fkdq" podStartSLOduration=3.220560057 podStartE2EDuration="23.835944883s" totalImagesPullingTime="20.615384826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:04.831277114 +0000 UTC m=+24.912718159" watchObservedRunningTime="2026-07-06 12:47:04.835944883 +0000 UTC m=+24.917385909" Jul 06 12:47:05.410266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.410046 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:05.410479 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.410046 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:05.410479 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:05.410372 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:05.410479 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.410049 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:05.410643 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:05.410416 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:05.410643 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:05.410537 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:05.625797 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.625773 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:47:05.626297 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.626157 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"ca2bec5e1b51b97cd823af7491a24213ee17857fb8b01d4aed97edbcf4c6a0b4"} Jul 06 12:47:05.627874 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.627797 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6z56m" event={"ID":"f004d9ed-be2a-4208-81a3-62fb8187f3c3","Type":"ContainerStarted","Data":"b6dd515330935498e756d5a1aa09f61a13be0532982990d89873f8a356c6af2d"} Jul 06 12:47:05.667827 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.667779 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6z56m" podStartSLOduration=4.236710634 podStartE2EDuration="24.667756659s" totalImagesPullingTime="20.431046025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:05.667240629 +0000 UTC m=+25.748681680" watchObservedRunningTime="2026-07-06 12:47:05.667756659 +0000 UTC m=+25.749197687" Jul 06 12:47:05.791021 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:05.790999 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 12:47:06.456242 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:06.456093 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T12:47:05.791017541Z","UUID":"4dd14790-2468-4c87-80f6-14c20c75fc53","Handler":null,"Name":"","Endpoint":""} Jul 06 12:47:06.458068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:06.458036 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 12:47:06.458263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:06.458097 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 12:47:06.632819 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:06.632746 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" event={"ID":"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7","Type":"ContainerStarted","Data":"6c4d904b9ac938749dcc4c0fde83dde8997a20cca5d171f4e11ef1c106434e9d"} Jul 06 12:47:06.637568 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:06.637537 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" event={"ID":"250d02a473ed274be5fbcb524906c9f6","Type":"ContainerStarted","Data":"eedce3f3a3035cd5ee6cecc7327e91b521f96d25e723c0610d860c03b224ff6d"} Jul 06 12:47:06.658074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:06.658017 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-21.ec2.internal" podStartSLOduration=25.657998436 podStartE2EDuration="25.657998436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:06.657213735 +0000 UTC m=+26.738654758" watchObservedRunningTime="2026-07-06 12:47:06.657998436 +0000 UTC m=+26.739439464" Jul 06 12:47:07.409659 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.409534 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:07.409659 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.409653 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:07.409896 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:07.409661 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:07.409896 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.409682 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:07.409896 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:07.409780 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:07.409896 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:07.409866 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:07.617847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.617797 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:47:07.618576 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.618553 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:47:07.642050 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.642015 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" event={"ID":"d958aea6-0cd4-4c1c-96eb-dc7a9149bab7","Type":"ContainerStarted","Data":"2bec9f9977de9fc49d071e68f77588b943a4e2eb150507fd05636e840b108f08"} Jul 06 12:47:07.645769 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.645740 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:47:07.646220 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:07.646198 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"f3eeb61b1d61a3e9274bd69d8578eff00bc6d7b792a3d3aa062aab0da6e78183"} Jul 06 12:47:09.409576 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.409340 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:09.410207 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.409369 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:09.410207 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:09.409627 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:09.410207 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.409408 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:09.410207 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:09.409691 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:09.410207 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:09.409765 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:09.652247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.652201 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ff1fa77-47ee-440b-9656-63ff8211a040" containerID="79dceb1d4524e2d5e48fc3d84eeec0974346d031492bd14268d72eb8bd36a0cc" exitCode=0 Jul 06 12:47:09.652426 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.652289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerDied","Data":"79dceb1d4524e2d5e48fc3d84eeec0974346d031492bd14268d72eb8bd36a0cc"} Jul 06 12:47:09.655866 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.655850 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:47:09.656214 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.656190 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"1e3f0cada5f425f1321c58d87fa4326cb725f49df5c12af7e133b2b1e2a26e3d"} Jul 06 12:47:09.656468 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.656447 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:47:09.656572 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.656511 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:47:09.656572 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.656559 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:47:09.656657 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.656649 2559 scope.go:117] "RemoveContainer" containerID="55abe16522ca83de366e10f64f7beefec1e528a00e9855faff4aba3e8722a62b" Jul 06 12:47:09.673135 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.673117 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:47:09.674142 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.674128 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:47:09.712291 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.712230 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dtpgh" podStartSLOduration=6.125690604 podStartE2EDuration="28.712214391s" totalImagesPullingTime="22.586523787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:07.664147034 +0000 UTC m=+27.745588060" watchObservedRunningTime="2026-07-06 12:47:09.712214391 +0000 UTC m=+29.793655418" Jul 06 12:47:09.877232 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.877200 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:47:09.877424 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.877345 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 12:47:09.877821 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:09.877807 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z4j4h" Jul 06 12:47:10.662402 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:10.662380 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:47:10.663077 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:10.662748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" event={"ID":"41e2b5df-fdbb-491d-962e-829c3f6f3405","Type":"ContainerStarted","Data":"5e7fc8c95919977d21c55448feb6c10b386d0336445cdd3225f7d95e9eaddc91"} Jul 06 12:47:10.697243 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:10.697202 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" podStartSLOduration=8.725059777 podStartE2EDuration="29.697184496s" totalImagesPullingTime="20.972124719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:10.695814528 +0000 UTC m=+30.777255574" watchObservedRunningTime="2026-07-06 12:47:10.697184496 +0000 UTC m=+30.778625518" Jul 06 12:47:11.114417 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.114188 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkfvl"] Jul 06 12:47:11.114576 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.114503 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:11.114639 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:11.114619 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:11.132180 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.132132 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bflvj"] Jul 06 12:47:11.132340 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.132302 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:11.132419 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:11.132401 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:11.148528 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.148502 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6xs84"] Jul 06 12:47:11.148650 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.148609 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:11.148701 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:11.148684 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:11.667336 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.667301 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ff1fa77-47ee-440b-9656-63ff8211a040" containerID="5da431e21fafe7efe9b85f28e45aebc703e2d453d9fec7ed0dc9d6c22d851649" exitCode=0 Jul 06 12:47:11.667809 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:11.667378 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerDied","Data":"5da431e21fafe7efe9b85f28e45aebc703e2d453d9fec7ed0dc9d6c22d851649"} Jul 06 12:47:12.409488 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:12.409463 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:12.409663 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:12.409571 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:13.092666 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.092635 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:13.093011 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.092696 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:13.093011 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.092774 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:47:13.093011 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.092779 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 12:47:13.093011 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.092826 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:45.092812437 +0000 UTC m=+65.174253443 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 12:47:13.093011 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.092838 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret podName:956e4c01-2aab-4200-8d08-b03df4427e55 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:45.092832719 +0000 UTC m=+65.174273724 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret") pod "global-pull-secret-syncer-jkfvl" (UID: "956e4c01-2aab-4200-8d08-b03df4427e55") : object "kube-system"/"original-pull-secret" not registered Jul 06 12:47:13.293783 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.293744 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:13.293969 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.293909 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 12:47:13.293969 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.293933 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 12:47:13.293969 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.293943 2559 projected.go:196] Error preparing data for projected volume kube-api-access-mff82 for pod openshift-network-diagnostics/network-check-target-6xs84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:47:13.294087 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.293993 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82 podName:5e8f204c-2f8a-4970-b041-8473338a77de nodeName:}" failed. No retries permitted until 2026-07-06 12:47:45.293980193 +0000 UTC m=+65.375421198 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mff82" (UniqueName: "kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82") pod "network-check-target-6xs84" (UID: "5e8f204c-2f8a-4970-b041-8473338a77de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 12:47:13.409816 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.409735 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:13.409962 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.409735 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:13.409962 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.409877 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:13.410035 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:13.409963 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:13.676264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.676159 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ff1fa77-47ee-440b-9656-63ff8211a040" containerID="095e7a11f5e19ca00142217b02e1101948c7d52505eb0a5b55269dbbf2889e02" exitCode=0 Jul 06 12:47:13.676264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:13.676203 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerDied","Data":"095e7a11f5e19ca00142217b02e1101948c7d52505eb0a5b55269dbbf2889e02"} Jul 06 12:47:14.410270 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:14.410143 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:14.410676 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:14.410290 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:15.410348 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:15.410305 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:15.410801 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:15.410310 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:15.410801 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:15.410420 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:15.410801 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:15.410512 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:16.410441 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:16.410401 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:16.410901 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:16.410533 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6xs84" podUID="5e8f204c-2f8a-4970-b041-8473338a77de" Jul 06 12:47:17.410029 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.409995 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:17.410249 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.410050 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:17.410249 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.410139 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bflvj" podUID="9b3622d5-e204-41c2-ba3b-a76597588b25" Jul 06 12:47:17.410384 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.410289 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkfvl" podUID="956e4c01-2aab-4200-8d08-b03df4427e55" Jul 06 12:47:17.692285 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.692211 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-21.ec2.internal" event="NodeReady" Jul 06 12:47:17.692687 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.692372 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 12:47:17.731244 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.731212 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78ccf75cfd-xps57"] Jul 06 12:47:17.733986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.733959 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.737817 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.737312 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 12:47:17.737817 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.737335 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 12:47:17.737817 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.737504 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 12:47:17.737817 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.737637 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-drj84\"" Jul 06 12:47:17.742349 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.742328 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 12:47:17.747327 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.747298 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78ccf75cfd-xps57"] Jul 06 12:47:17.750399 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.750375 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-stdbh"] Jul 06 12:47:17.752228 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.752210 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.756123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.756096 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 12:47:17.756240 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.756123 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9njlh\"" Jul 06 12:47:17.756240 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.756215 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 12:47:17.758403 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.758385 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2ptv2"] Jul 06 12:47:17.760115 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.760098 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:17.762664 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.762644 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 12:47:17.762769 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.762717 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 12:47:17.762862 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.762842 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-28thq\"" Jul 06 12:47:17.762962 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.762899 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 12:47:17.767511 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.767406 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-stdbh"] Jul 06 12:47:17.783716 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.783689 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2ptv2"] Jul 06 12:47:17.828364 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828327 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16edb0eb-f728-4318-9245-e64587cb9253-ca-trust-extracted\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.828553 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828375 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.828553 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-bound-sa-token\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.828553 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828437 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmglj\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-kube-api-access-hmglj\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.828553 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828494 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6990e881-e5ad-4208-9548-3ccabbff5f89-tmp-dir\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.828553 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828534 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-trusted-ca\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.828811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828558 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6990e881-e5ad-4208-9548-3ccabbff5f89-config-volume\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.828811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828581 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.828811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828600 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-registry-certificates\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.828811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828623 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpf4q\" (UniqueName: \"kubernetes.io/projected/6990e881-e5ad-4208-9548-3ccabbff5f89-kube-api-access-gpf4q\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.828811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828718 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-installation-pull-secrets\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.829059 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.828802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-image-registry-private-configuration\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.925640 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.925604 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z4j4h_850b8299-9c34-4e86-9d54-95161af94888/konnectivity-agent/0.log" limit=1024 Jul 06 12:47:17.929580 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929552 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-image-registry-private-configuration\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.929691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929603 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16edb0eb-f728-4318-9245-e64587cb9253-ca-trust-extracted\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.929691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929624 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.929691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929646 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-bound-sa-token\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.929691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmglj\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-kube-api-access-hmglj\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.929880 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6990e881-e5ad-4208-9548-3ccabbff5f89-tmp-dir\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.929880 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-trusted-ca\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.929880 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.929766 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:17.929880 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.929788 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:17.929880 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.929848 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:18.4298289 +0000 UTC m=+38.511269925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:17.930372 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930348 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16edb0eb-f728-4318-9245-e64587cb9253-ca-trust-extracted\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.930473 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.929768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9dl\" (UniqueName: \"kubernetes.io/projected/8f0c1232-a241-4386-9424-8c3011559d76-kube-api-access-cn9dl\") pod \"ingress-canary-2ptv2\" (UID: \"8f0c1232-a241-4386-9424-8c3011559d76\") " pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:17.930473 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930419 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6990e881-e5ad-4208-9548-3ccabbff5f89-config-volume\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.930473 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.930611 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-registry-certificates\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.930611 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpf4q\" (UniqueName: \"kubernetes.io/projected/6990e881-e5ad-4208-9548-3ccabbff5f89-kube-api-access-gpf4q\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.930611 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930513 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6990e881-e5ad-4208-9548-3ccabbff5f89-tmp-dir\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.930611 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f0c1232-a241-4386-9424-8c3011559d76-cert\") pod \"ingress-canary-2ptv2\" (UID: \"8f0c1232-a241-4386-9424-8c3011559d76\") " pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:17.930770 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.930628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-installation-pull-secrets\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.931066 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.931020 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6990e881-e5ad-4208-9548-3ccabbff5f89-config-volume\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.931148 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.931066 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:17.931148 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:17.931120 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:18.431103095 +0000 UTC m=+38.512544119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:17.932511 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.931342 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-trusted-ca\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.932511 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.931514 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-registry-certificates\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.934939 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.934885 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-installation-pull-secrets\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.935319 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.935298 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-image-registry-private-configuration\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.940094 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.940069 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-bound-sa-token\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:17.940233 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.940210 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpf4q\" (UniqueName: \"kubernetes.io/projected/6990e881-e5ad-4208-9548-3ccabbff5f89-kube-api-access-gpf4q\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:17.940233 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:17.940219 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmglj\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-kube-api-access-hmglj\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:18.031886 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.031695 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f0c1232-a241-4386-9424-8c3011559d76-cert\") pod \"ingress-canary-2ptv2\" (UID: \"8f0c1232-a241-4386-9424-8c3011559d76\") " pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:18.032017 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.031953 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9dl\" (UniqueName: \"kubernetes.io/projected/8f0c1232-a241-4386-9424-8c3011559d76-kube-api-access-cn9dl\") pod \"ingress-canary-2ptv2\" (UID: \"8f0c1232-a241-4386-9424-8c3011559d76\") " pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:18.034851 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.034826 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f0c1232-a241-4386-9424-8c3011559d76-cert\") pod \"ingress-canary-2ptv2\" (UID: \"8f0c1232-a241-4386-9424-8c3011559d76\") " pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:18.040805 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.040762 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-z4j4h_850b8299-9c34-4e86-9d54-95161af94888/konnectivity-agent/0.log" limit=1024 Jul 06 12:47:18.041733 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.041710 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9dl\" (UniqueName: \"kubernetes.io/projected/8f0c1232-a241-4386-9424-8c3011559d76-kube-api-access-cn9dl\") pod \"ingress-canary-2ptv2\" (UID: \"8f0c1232-a241-4386-9424-8c3011559d76\") " pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:18.071928 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.071904 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2ptv2" Jul 06 12:47:18.205082 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.205015 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2ptv2"] Jul 06 12:47:18.208315 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:47:18.208281 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0c1232_a241_4386_9424_8c3011559d76.slice/crio-fcb8268143d1142727ee9d5f14f15ca79628287b2a30d06f7fc586e287279a4d WatchSource:0}: Error finding container fcb8268143d1142727ee9d5f14f15ca79628287b2a30d06f7fc586e287279a4d: Status 404 returned error can't find the container with id fcb8268143d1142727ee9d5f14f15ca79628287b2a30d06f7fc586e287279a4d Jul 06 12:47:18.412893 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.412866 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:18.415647 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.415614 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 12:47:18.415798 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.415683 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mmbsd\"" Jul 06 12:47:18.415798 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.415616 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 12:47:18.435418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.435391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:18.435547 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.435436 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:18.435547 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:18.435533 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:18.435617 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:18.435544 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:18.435617 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:18.435560 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:18.435617 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:18.435581 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:19.435567601 +0000 UTC m=+39.517008606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:18.435617 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:18.435602 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:19.435589078 +0000 UTC m=+39.517030083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:18.694378 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.694286 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ff1fa77-47ee-440b-9656-63ff8211a040" containerID="e4340150822c7f72a2df7214863b38bba2e5cf09a20cf1030646dc150052c544" exitCode=0 Jul 06 12:47:18.695023 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.694375 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerDied","Data":"e4340150822c7f72a2df7214863b38bba2e5cf09a20cf1030646dc150052c544"} Jul 06 12:47:18.695455 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:18.695331 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2ptv2" event={"ID":"8f0c1232-a241-4386-9424-8c3011559d76","Type":"ContainerStarted","Data":"fcb8268143d1142727ee9d5f14f15ca79628287b2a30d06f7fc586e287279a4d"} Jul 06 12:47:19.409779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.409749 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:19.410417 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.409753 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:19.413094 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.412908 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 12:47:19.413886 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.413861 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dxdc8\"" Jul 06 12:47:19.413975 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.413883 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 12:47:19.445988 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.445963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:19.446116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.446023 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:19.446116 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:19.446111 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:19.446229 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:19.446124 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:19.446229 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:19.446189 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:21.446159233 +0000 UTC m=+41.527600243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:19.446229 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:19.446193 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:19.446359 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:19.446263 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:21.446244573 +0000 UTC m=+41.527685592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:19.701879 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.701790 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ff1fa77-47ee-440b-9656-63ff8211a040" containerID="ecb3bacb3231c8b6d97c50b9959dd44b3aad6b01762000df91c08219815dde15" exitCode=0 Jul 06 12:47:19.701879 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:19.701836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerDied","Data":"ecb3bacb3231c8b6d97c50b9959dd44b3aad6b01762000df91c08219815dde15"} Jul 06 12:47:20.712096 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:20.711886 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" event={"ID":"9ff1fa77-47ee-440b-9656-63ff8211a040","Type":"ContainerStarted","Data":"dc1eaabfe019e6af5abb2be64b84d0630714473a5e7e2fdd654b520c0d146f5b"} Jul 06 12:47:20.742091 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:20.741883 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ngzm5" podStartSLOduration=8.113569568 podStartE2EDuration="39.741867361s" totalImagesPullingTime="31.628297793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:20.741212104 +0000 UTC m=+40.822653132" watchObservedRunningTime="2026-07-06 12:47:20.741867361 +0000 UTC m=+40.823308387" Jul 06 12:47:21.462007 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:21.461972 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:21.462199 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:21.462057 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:21.462199 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:21.462130 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:21.462199 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:21.462159 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:21.462199 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:21.462196 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:21.462351 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:21.462226 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:25.462206832 +0000 UTC m=+45.543647857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:21.462351 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:21.462244 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:25.462236106 +0000 UTC m=+45.543677110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:21.716188 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:21.716075 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2ptv2" event={"ID":"8f0c1232-a241-4386-9424-8c3011559d76","Type":"ContainerStarted","Data":"68ecd6d33a26af6ab95efd90bd51241e9559a8926fbc883bbfe82e5754d0709a"} Jul 06 12:47:21.732784 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:21.732739 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2ptv2" podStartSLOduration=2.081144607 podStartE2EDuration="4.732723095s" totalImagesPullingTime="2.651578488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:21.732638089 +0000 UTC m=+41.814079140" watchObservedRunningTime="2026-07-06 12:47:21.732723095 +0000 UTC m=+41.814164121" Jul 06 12:47:25.493662 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:25.493617 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:25.494040 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:25.493677 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:25.494040 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:25.493779 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:25.494040 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:25.493787 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:25.494040 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:25.493803 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:25.494040 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:25.493840 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:33.493826511 +0000 UTC m=+53.575267516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:25.494040 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:25.493862 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:33.493848767 +0000 UTC m=+53.575289775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:33.546182 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:33.546141 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:33.546613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:33.546207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:33.546613 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:33.546299 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:33.546613 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:33.546313 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:33.546613 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:33.546338 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:33.546613 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:33.546359 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:49.546344803 +0000 UTC m=+69.627785808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:33.546613 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:33.546390 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:47:49.54637207 +0000 UTC m=+69.627813082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:41.678206 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:41.678178 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tgmrj" Jul 06 12:47:45.127101 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.127057 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:47:45.127566 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.127116 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:45.133049 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.133026 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 12:47:45.133344 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.133329 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 12:47:45.137962 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:45.137942 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 12:47:45.138031 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:45.138014 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs podName:9b3622d5-e204-41c2-ba3b-a76597588b25 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:49.137997458 +0000 UTC m=+129.219438468 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs") pod "network-metrics-daemon-bflvj" (UID: "9b3622d5-e204-41c2-ba3b-a76597588b25") : secret "metrics-daemon-secret" not found Jul 06 12:47:45.141095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.141066 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/956e4c01-2aab-4200-8d08-b03df4427e55-original-pull-secret\") pod \"global-pull-secret-syncer-jkfvl\" (UID: \"956e4c01-2aab-4200-8d08-b03df4427e55\") " pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:45.226463 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.226420 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkfvl" Jul 06 12:47:45.328939 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.328911 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:45.331701 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.331676 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 12:47:45.342640 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.342614 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkfvl"] Jul 06 12:47:45.343489 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.343475 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 12:47:45.345197 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:47:45.345161 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956e4c01_2aab_4200_8d08_b03df4427e55.slice/crio-0efb8739da20b4c9a573fcb93821e9dfaa1df990d9275e9a709cc559349ddcb9 WatchSource:0}: Error finding container 0efb8739da20b4c9a573fcb93821e9dfaa1df990d9275e9a709cc559349ddcb9: Status 404 returned error can't find the container with id 0efb8739da20b4c9a573fcb93821e9dfaa1df990d9275e9a709cc559349ddcb9 Jul 06 12:47:45.352825 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.352805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mff82\" (UniqueName: \"kubernetes.io/projected/5e8f204c-2f8a-4970-b041-8473338a77de-kube-api-access-mff82\") pod \"network-check-target-6xs84\" (UID: \"5e8f204c-2f8a-4970-b041-8473338a77de\") " pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:45.424969 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.424947 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mmbsd\"" Jul 06 12:47:45.432384 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.432366 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:45.551579 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.551550 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6xs84"] Jul 06 12:47:45.555512 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:47:45.555490 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8f204c_2f8a_4970_b041_8473338a77de.slice/crio-2dc0b82b7ad9dccf7c63c1c2bab72dd148b1d3fc518b2c9bea1f19840fb7c3dc WatchSource:0}: Error finding container 2dc0b82b7ad9dccf7c63c1c2bab72dd148b1d3fc518b2c9bea1f19840fb7c3dc: Status 404 returned error can't find the container with id 2dc0b82b7ad9dccf7c63c1c2bab72dd148b1d3fc518b2c9bea1f19840fb7c3dc Jul 06 12:47:45.772631 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.772538 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6xs84" event={"ID":"5e8f204c-2f8a-4970-b041-8473338a77de","Type":"ContainerStarted","Data":"2dc0b82b7ad9dccf7c63c1c2bab72dd148b1d3fc518b2c9bea1f19840fb7c3dc"} Jul 06 12:47:45.773543 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:45.773515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkfvl" event={"ID":"956e4c01-2aab-4200-8d08-b03df4427e55","Type":"ContainerStarted","Data":"0efb8739da20b4c9a573fcb93821e9dfaa1df990d9275e9a709cc559349ddcb9"} Jul 06 12:47:49.559340 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:49.559301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:47:49.559772 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:49.559403 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:47:49.559772 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:49.559454 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:47:49.559772 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:49.559518 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:47:49.559772 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:49.559532 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:47:49.559772 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:49.559549 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:21.559525373 +0000 UTC m=+101.640966394 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:47:49.559772 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:47:49.559585 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:21.559567965 +0000 UTC m=+101.641008975 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:47:51.791317 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:51.791287 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6xs84" event={"ID":"5e8f204c-2f8a-4970-b041-8473338a77de","Type":"ContainerStarted","Data":"57abd6c94f699f82ef869c4a76fb9bc1413ac2476f586a77069309cece3d2aa5"} Jul 06 12:47:51.791777 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:51.791403 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:47:51.792935 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:51.792910 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkfvl" event={"ID":"956e4c01-2aab-4200-8d08-b03df4427e55","Type":"ContainerStarted","Data":"a6730ab2c6e7fc9ecb19c9cd62321ebac3262da9a2d3e547e66aa20feb0e635e"} Jul 06 12:47:51.809842 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:51.809608 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6xs84" podStartSLOduration=65.605562727 podStartE2EDuration="1m10.809590306s" totalImagesPullingTime="5.204027579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:51.809140853 +0000 UTC m=+71.890581880" watchObservedRunningTime="2026-07-06 12:47:51.809590306 +0000 UTC m=+71.891031333" Jul 06 12:47:51.829072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:47:51.829020 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jkfvl" podStartSLOduration=65.402756245 podStartE2EDuration="1m10.829008111s" totalImagesPullingTime="5.426251866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:47:51.828523648 +0000 UTC m=+71.909964674" watchObservedRunningTime="2026-07-06 12:47:51.829008111 +0000 UTC m=+71.910449137" Jul 06 12:48:21.585939 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:21.585891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") pod \"image-registry-78ccf75cfd-xps57\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:48:21.585939 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:21.585948 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:48:21.586438 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:21.586056 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 12:48:21.586438 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:21.586063 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 12:48:21.586438 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:21.586080 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ccf75cfd-xps57: secret "image-registry-tls" not found Jul 06 12:48:21.586438 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:21.586117 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls podName:6990e881-e5ad-4208-9548-3ccabbff5f89 nodeName:}" failed. No retries permitted until 2026-07-06 12:49:25.586103336 +0000 UTC m=+165.667544341 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls") pod "dns-default-stdbh" (UID: "6990e881-e5ad-4208-9548-3ccabbff5f89") : secret "dns-default-metrics-tls" not found Jul 06 12:48:21.586438 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:21.586130 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls podName:16edb0eb-f728-4318-9245-e64587cb9253 nodeName:}" failed. No retries permitted until 2026-07-06 12:49:25.586124312 +0000 UTC m=+165.667565317 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls") pod "image-registry-78ccf75cfd-xps57" (UID: "16edb0eb-f728-4318-9245-e64587cb9253") : secret "image-registry-tls" not found Jul 06 12:48:22.797634 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:22.797609 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6xs84" Jul 06 12:48:23.640724 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.640692 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d"] Jul 06 12:48:23.643258 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.643231 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.646385 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.646362 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 12:48:23.646919 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.646900 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 12:48:23.649578 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.649557 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 12:48:23.649651 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.649627 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 12:48:23.649732 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.649714 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lcdv5\"" Jul 06 12:48:23.659702 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.659672 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d"] Jul 06 12:48:23.699751 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.699707 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/73d957ac-833e-4def-a90a-0f720294def7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.699751 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.699750 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr2zf\" (UniqueName: \"kubernetes.io/projected/73d957ac-833e-4def-a90a-0f720294def7-kube-api-access-jr2zf\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.699957 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.699824 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.716303 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.716261 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4"] Jul 06 12:48:23.717917 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.717902 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" Jul 06 12:48:23.720828 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.720808 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hdzj8\"" Jul 06 12:48:23.741445 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.741420 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-v9vm9"] Jul 06 12:48:23.743104 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.743090 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.746505 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.746476 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4"] Jul 06 12:48:23.748543 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.748520 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 12:48:23.748823 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.748796 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-d5v97\"" Jul 06 12:48:23.748906 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.748885 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 12:48:23.748976 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.748892 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 12:48:23.750014 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.749927 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 12:48:23.760078 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.760058 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 12:48:23.773158 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.773132 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-v9vm9"] Jul 06 12:48:23.800778 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800745 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cbacad8-eb42-4efc-81c1-a49709150841-config\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.800778 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800784 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99zcx\" (UniqueName: \"kubernetes.io/projected/0b9a1134-254c-4fdd-897f-0d000d335319-kube-api-access-99zcx\") pod \"network-check-source-989f8b8dc-2lxt4\" (UID: \"0b9a1134-254c-4fdd-897f-0d000d335319\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800817 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlgcs\" (UniqueName: \"kubernetes.io/projected/9cbacad8-eb42-4efc-81c1-a49709150841-kube-api-access-dlgcs\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cbacad8-eb42-4efc-81c1-a49709150841-trusted-ca\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800941 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cbacad8-eb42-4efc-81c1-a49709150841-serving-cert\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.800986 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/73d957ac-833e-4def-a90a-0f720294def7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:23.800998 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.801013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr2zf\" (UniqueName: \"kubernetes.io/projected/73d957ac-833e-4def-a90a-0f720294def7-kube-api-access-jr2zf\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.801310 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:23.801060 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls podName:73d957ac-833e-4def-a90a-0f720294def7 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:24.301041703 +0000 UTC m=+104.382482715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fr49d" (UID: "73d957ac-833e-4def-a90a-0f720294def7") : secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:23.801733 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.801712 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/73d957ac-833e-4def-a90a-0f720294def7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.814374 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.814342 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr2zf\" (UniqueName: \"kubernetes.io/projected/73d957ac-833e-4def-a90a-0f720294def7-kube-api-access-jr2zf\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:23.902307 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.902225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlgcs\" (UniqueName: \"kubernetes.io/projected/9cbacad8-eb42-4efc-81c1-a49709150841-kube-api-access-dlgcs\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.902307 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.902261 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cbacad8-eb42-4efc-81c1-a49709150841-trusted-ca\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.902533 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.902453 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cbacad8-eb42-4efc-81c1-a49709150841-serving-cert\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.902533 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.902519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cbacad8-eb42-4efc-81c1-a49709150841-config\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.902660 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.902542 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99zcx\" (UniqueName: \"kubernetes.io/projected/0b9a1134-254c-4fdd-897f-0d000d335319-kube-api-access-99zcx\") pod \"network-check-source-989f8b8dc-2lxt4\" (UID: \"0b9a1134-254c-4fdd-897f-0d000d335319\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" Jul 06 12:48:23.903112 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.903087 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9cbacad8-eb42-4efc-81c1-a49709150841-config\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.903234 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.903153 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cbacad8-eb42-4efc-81c1-a49709150841-trusted-ca\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.904750 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.904733 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9cbacad8-eb42-4efc-81c1-a49709150841-serving-cert\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:23.915759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.915727 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99zcx\" (UniqueName: \"kubernetes.io/projected/0b9a1134-254c-4fdd-897f-0d000d335319-kube-api-access-99zcx\") pod \"network-check-source-989f8b8dc-2lxt4\" (UID: \"0b9a1134-254c-4fdd-897f-0d000d335319\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" Jul 06 12:48:23.916825 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:23.916805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlgcs\" (UniqueName: \"kubernetes.io/projected/9cbacad8-eb42-4efc-81c1-a49709150841-kube-api-access-dlgcs\") pod \"console-operator-68b757fb67-v9vm9\" (UID: \"9cbacad8-eb42-4efc-81c1-a49709150841\") " pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:24.026332 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.026286 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" Jul 06 12:48:24.055246 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.055212 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:24.163348 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.163244 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4"] Jul 06 12:48:24.166213 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:24.166191 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9a1134_254c_4fdd_897f_0d000d335319.slice/crio-bb5b3d2965af773019cb298491647a56baec8b291fb11ed630b3695be2507525 WatchSource:0}: Error finding container bb5b3d2965af773019cb298491647a56baec8b291fb11ed630b3695be2507525: Status 404 returned error can't find the container with id bb5b3d2965af773019cb298491647a56baec8b291fb11ed630b3695be2507525 Jul 06 12:48:24.189820 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.189793 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-v9vm9"] Jul 06 12:48:24.192365 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:24.192340 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cbacad8_eb42_4efc_81c1_a49709150841.slice/crio-68aea96f039148bab32f00ba83e8aa9aff8e00486df7e4b97d9939d456f71bd4 WatchSource:0}: Error finding container 68aea96f039148bab32f00ba83e8aa9aff8e00486df7e4b97d9939d456f71bd4: Status 404 returned error can't find the container with id 68aea96f039148bab32f00ba83e8aa9aff8e00486df7e4b97d9939d456f71bd4 Jul 06 12:48:24.304878 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.304842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:24.305034 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:24.304975 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:24.305034 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:24.305032 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls podName:73d957ac-833e-4def-a90a-0f720294def7 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:25.305018421 +0000 UTC m=+105.386459430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fr49d" (UID: "73d957ac-833e-4def-a90a-0f720294def7") : secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:24.873784 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.873715 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" event={"ID":"9cbacad8-eb42-4efc-81c1-a49709150841","Type":"ContainerStarted","Data":"68aea96f039148bab32f00ba83e8aa9aff8e00486df7e4b97d9939d456f71bd4"} Jul 06 12:48:24.875316 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.875287 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" event={"ID":"0b9a1134-254c-4fdd-897f-0d000d335319","Type":"ContainerStarted","Data":"d1fbb735f11704f8024fd696a97c907d22dd0bce70f791ee2be2df5f01913ae8"} Jul 06 12:48:24.875491 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.875335 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" event={"ID":"0b9a1134-254c-4fdd-897f-0d000d335319","Type":"ContainerStarted","Data":"bb5b3d2965af773019cb298491647a56baec8b291fb11ed630b3695be2507525"} Jul 06 12:48:24.899910 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:24.899858 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-2lxt4" podStartSLOduration=1.899841449 podStartE2EDuration="1.899841449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:24.89974149 +0000 UTC m=+104.981182519" watchObservedRunningTime="2026-07-06 12:48:24.899841449 +0000 UTC m=+104.981282475" Jul 06 12:48:25.312421 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:25.312280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:25.312421 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:25.312407 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:25.312672 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:25.312468 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls podName:73d957ac-833e-4def-a90a-0f720294def7 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:27.312451987 +0000 UTC m=+107.393892996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fr49d" (UID: "73d957ac-833e-4def-a90a-0f720294def7") : secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:26.883393 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:26.883367 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/0.log" Jul 06 12:48:26.883791 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:26.883409 2559 generic.go:356] "Generic (PLEG): container finished" podID="9cbacad8-eb42-4efc-81c1-a49709150841" containerID="5a0ff625903fc5a3d15a09c5d4244d81abddf8c5bdb4592e498c5bcb79f2aac0" exitCode=255 Jul 06 12:48:26.883791 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:26.883445 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" event={"ID":"9cbacad8-eb42-4efc-81c1-a49709150841","Type":"ContainerDied","Data":"5a0ff625903fc5a3d15a09c5d4244d81abddf8c5bdb4592e498c5bcb79f2aac0"} Jul 06 12:48:26.883791 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:26.883677 2559 scope.go:117] "RemoveContainer" containerID="5a0ff625903fc5a3d15a09c5d4244d81abddf8c5bdb4592e498c5bcb79f2aac0" Jul 06 12:48:27.330105 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.330061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:27.330305 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:27.330241 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:27.330357 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:27.330330 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls podName:73d957ac-833e-4def-a90a-0f720294def7 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:31.330312415 +0000 UTC m=+111.411753420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fr49d" (UID: "73d957ac-833e-4def-a90a-0f720294def7") : secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:27.888402 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.888372 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:48:27.888836 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.888792 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/0.log" Jul 06 12:48:27.888925 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.888847 2559 generic.go:356] "Generic (PLEG): container finished" podID="9cbacad8-eb42-4efc-81c1-a49709150841" containerID="928ec9ce102442891f1c7a6c57db347d0936297e124318051b06eabf0b1ecb2d" exitCode=255 Jul 06 12:48:27.888925 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.888913 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" event={"ID":"9cbacad8-eb42-4efc-81c1-a49709150841","Type":"ContainerDied","Data":"928ec9ce102442891f1c7a6c57db347d0936297e124318051b06eabf0b1ecb2d"} Jul 06 12:48:27.888997 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.888990 2559 scope.go:117] "RemoveContainer" containerID="5a0ff625903fc5a3d15a09c5d4244d81abddf8c5bdb4592e498c5bcb79f2aac0" Jul 06 12:48:27.889241 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:27.889224 2559 scope.go:117] "RemoveContainer" containerID="928ec9ce102442891f1c7a6c57db347d0936297e124318051b06eabf0b1ecb2d" Jul 06 12:48:27.889452 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:27.889433 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-v9vm9_openshift-console-operator(9cbacad8-eb42-4efc-81c1-a49709150841)\"" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" podUID="9cbacad8-eb42-4efc-81c1-a49709150841" Jul 06 12:48:28.193069 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.193028 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-48wjq"] Jul 06 12:48:28.195684 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.195663 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.237317 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.237280 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 12:48:28.237317 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.237316 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 12:48:28.240846 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.240829 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 12:48:28.261015 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.260992 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 12:48:28.261405 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.261389 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h754f\"" Jul 06 12:48:28.339701 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.339666 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-data-volume\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.339701 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.339700 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.339923 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.339765 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnfmc\" (UniqueName: \"kubernetes.io/projected/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-api-access-gnfmc\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.339923 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.339814 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.339923 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.339846 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-crio-socket\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.434363 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.434332 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-48wjq"] Jul 06 12:48:28.440451 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-data-volume\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440590 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440590 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440494 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnfmc\" (UniqueName: \"kubernetes.io/projected/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-api-access-gnfmc\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440590 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440575 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440729 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-crio-socket\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440729 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.440664 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 12:48:28.440729 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.440698 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 12:48:28.440847 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.440732 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:28.940715116 +0000 UTC m=+109.022156125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : secret "insights-runtime-extractor-tls" not found Jul 06 12:48:28.440847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440732 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-data-volume\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.440708 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-crio-socket\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.440847 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.440784 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:28.940766172 +0000 UTC m=+109.022207194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : configmap "kube-rbac-proxy" not found Jul 06 12:48:28.451382 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.451319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnfmc\" (UniqueName: \"kubernetes.io/projected/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-api-access-gnfmc\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.893420 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.893350 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:48:28.893758 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.893666 2559 scope.go:117] "RemoveContainer" containerID="928ec9ce102442891f1c7a6c57db347d0936297e124318051b06eabf0b1ecb2d" Jul 06 12:48:28.893845 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.893825 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-v9vm9_openshift-console-operator(9cbacad8-eb42-4efc-81c1-a49709150841)\"" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" podUID="9cbacad8-eb42-4efc-81c1-a49709150841" Jul 06 12:48:28.943922 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.943890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.944060 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:28.943964 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:28.944060 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.944024 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 12:48:28.944060 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.944041 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 12:48:28.944192 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.944081 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:29.944065152 +0000 UTC m=+110.025506157 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : secret "insights-runtime-extractor-tls" not found Jul 06 12:48:28.944192 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:28.944099 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:29.944087368 +0000 UTC m=+110.025528373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : configmap "kube-rbac-proxy" not found Jul 06 12:48:29.951837 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:29.951796 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:29.952246 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:29.951942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:29.952246 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:29.951954 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 12:48:29.952246 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:29.952018 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:31.951999003 +0000 UTC m=+112.033440028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : secret "insights-runtime-extractor-tls" not found Jul 06 12:48:29.952246 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:29.952034 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 12:48:29.952246 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:29.952073 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:31.952062374 +0000 UTC m=+112.033503380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : configmap "kube-rbac-proxy" not found Jul 06 12:48:30.951838 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:30.951811 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4sns4_3b1b9765-789d-4fc8-bb16-10601e312f90/dns-node-resolver/0.log" Jul 06 12:48:31.365092 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:31.364998 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:31.365468 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:31.365151 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:31.365468 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:31.365235 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls podName:73d957ac-833e-4def-a90a-0f720294def7 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:39.36521976 +0000 UTC m=+119.446660770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fr49d" (UID: "73d957ac-833e-4def-a90a-0f720294def7") : secret "cluster-monitoring-operator-tls" not found Jul 06 12:48:31.942296 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:31.942272 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6vxs7_a1cb7806-30f8-40b1-9cff-0dc679136c5a/node-ca/0.log" Jul 06 12:48:31.970617 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:31.970586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:31.970715 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:31.970657 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:31.970756 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:31.970731 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 12:48:31.970756 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:31.970750 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 12:48:31.970816 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:31.970794 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:35.970778356 +0000 UTC m=+116.052219363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : secret "insights-runtime-extractor-tls" not found Jul 06 12:48:31.970816 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:31.970808 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:35.97080128 +0000 UTC m=+116.052242285 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : configmap "kube-rbac-proxy" not found Jul 06 12:48:32.772914 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:32.772887 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2ptv2_8f0c1232-a241-4386-9424-8c3011559d76/serve-healthcheck-canary/0.log" Jul 06 12:48:34.055893 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:34.055841 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:34.056296 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:34.055920 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:34.056296 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:34.056283 2559 scope.go:117] "RemoveContainer" containerID="928ec9ce102442891f1c7a6c57db347d0936297e124318051b06eabf0b1ecb2d" Jul 06 12:48:34.056445 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:34.056426 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-v9vm9_openshift-console-operator(9cbacad8-eb42-4efc-81c1-a49709150841)\"" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" podUID="9cbacad8-eb42-4efc-81c1-a49709150841" Jul 06 12:48:36.005504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:36.005459 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:36.005883 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:36.005532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:36.005883 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:36.005603 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 12:48:36.005883 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:36.005652 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:44.005638235 +0000 UTC m=+124.087079240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : configmap "kube-rbac-proxy" not found Jul 06 12:48:36.005883 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:36.005603 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 12:48:36.005883 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:48:36.005683 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls podName:f82becb4-8b89-4e3e-8687-eb04cc2c6b73 nodeName:}" failed. No retries permitted until 2026-07-06 12:48:44.005676924 +0000 UTC m=+124.087117928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48wjq" (UID: "f82becb4-8b89-4e3e-8687-eb04cc2c6b73") : secret "insights-runtime-extractor-tls" not found Jul 06 12:48:39.434726 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:39.434682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:39.436856 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:39.436834 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/73d957ac-833e-4def-a90a-0f720294def7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fr49d\" (UID: \"73d957ac-833e-4def-a90a-0f720294def7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:39.552601 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:39.552571 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" Jul 06 12:48:39.688384 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:39.688304 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d"] Jul 06 12:48:39.691871 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:39.691845 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d957ac_833e_4def_a90a_0f720294def7.slice/crio-32a702ea0c24b5760464c9c9994b19387ef7b900f83e6712f2f0d5ba07b55146 WatchSource:0}: Error finding container 32a702ea0c24b5760464c9c9994b19387ef7b900f83e6712f2f0d5ba07b55146: Status 404 returned error can't find the container with id 32a702ea0c24b5760464c9c9994b19387ef7b900f83e6712f2f0d5ba07b55146 Jul 06 12:48:39.925815 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:39.925772 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" event={"ID":"73d957ac-833e-4def-a90a-0f720294def7","Type":"ContainerStarted","Data":"32a702ea0c24b5760464c9c9994b19387ef7b900f83e6712f2f0d5ba07b55146"} Jul 06 12:48:41.933269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:41.933228 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" event={"ID":"73d957ac-833e-4def-a90a-0f720294def7","Type":"ContainerStarted","Data":"c8e6169e341d10d58b3a0e18ccea86b4dd1955e3d4f458c78ad642985477b633"} Jul 06 12:48:41.962121 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:41.962074 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fr49d" podStartSLOduration=16.905649512 podStartE2EDuration="18.962059399s" totalImagesPullingTime="2.056409887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:41.961422345 +0000 UTC m=+122.042863372" watchObservedRunningTime="2026-07-06 12:48:41.962059399 +0000 UTC m=+122.043500426" Jul 06 12:48:44.071998 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.071959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:44.072373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.072020 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:44.072479 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.072463 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:44.074101 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.074071 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f82becb4-8b89-4e3e-8687-eb04cc2c6b73-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48wjq\" (UID: \"f82becb4-8b89-4e3e-8687-eb04cc2c6b73\") " pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:44.106324 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.106304 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h754f\"" Jul 06 12:48:44.114051 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.114037 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-48wjq" Jul 06 12:48:44.254589 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:44.254546 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82becb4_8b89_4e3e_8687_eb04cc2c6b73.slice/crio-7c65c45865f13149603cd1ea662164e8e94a501448350a20b9141841eb7d4343 WatchSource:0}: Error finding container 7c65c45865f13149603cd1ea662164e8e94a501448350a20b9141841eb7d4343: Status 404 returned error can't find the container with id 7c65c45865f13149603cd1ea662164e8e94a501448350a20b9141841eb7d4343 Jul 06 12:48:44.258572 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.258552 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-48wjq"] Jul 06 12:48:44.944529 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:44.944402 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48wjq" event={"ID":"f82becb4-8b89-4e3e-8687-eb04cc2c6b73","Type":"ContainerStarted","Data":"7c65c45865f13149603cd1ea662164e8e94a501448350a20b9141841eb7d4343"} Jul 06 12:48:46.410717 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:46.410683 2559 scope.go:117] "RemoveContainer" containerID="928ec9ce102442891f1c7a6c57db347d0936297e124318051b06eabf0b1ecb2d" Jul 06 12:48:48.961709 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.961684 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:48:48.962086 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.961775 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" event={"ID":"9cbacad8-eb42-4efc-81c1-a49709150841","Type":"ContainerStarted","Data":"c07e5737b3a90bcd599ca20904ea0e6c40f8751f79106b39e93ce27383ab8aab"} Jul 06 12:48:48.962086 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.962061 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:48.963584 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.963552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48wjq" event={"ID":"f82becb4-8b89-4e3e-8687-eb04cc2c6b73","Type":"ContainerStarted","Data":"a411eb57798cd8b05b9d333c58344da591624e0eaf6a0afae9a62375e3673a83"} Jul 06 12:48:48.963699 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.963605 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48wjq" event={"ID":"f82becb4-8b89-4e3e-8687-eb04cc2c6b73","Type":"ContainerStarted","Data":"17ce2fcb16526967a627b14efc7e94f9786c7ff96bccb32c4d86b19012cd338c"} Jul 06 12:48:48.966809 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.966784 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" Jul 06 12:48:48.990820 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:48.990780 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-v9vm9" podStartSLOduration=23.442708257 podStartE2EDuration="25.990767155s" totalImagesPullingTime="2.548058898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:48.989564823 +0000 UTC m=+129.071005878" watchObservedRunningTime="2026-07-06 12:48:48.990767155 +0000 UTC m=+129.072208181" Jul 06 12:48:49.209440 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:49.209401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:48:49.212261 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:49.212198 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b3622d5-e204-41c2-ba3b-a76597588b25-metrics-certs\") pod \"network-metrics-daemon-bflvj\" (UID: \"9b3622d5-e204-41c2-ba3b-a76597588b25\") " pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:48:49.424379 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:49.424348 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dxdc8\"" Jul 06 12:48:49.432560 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:49.432538 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bflvj" Jul 06 12:48:49.571746 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:49.571718 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bflvj"] Jul 06 12:48:49.575426 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:49.575396 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3622d5_e204_41c2_ba3b_a76597588b25.slice/crio-b9d0b9ca47ac965c8638771668b1b24fd0f39bba11ec192d65372bc3e8b3ed2d WatchSource:0}: Error finding container b9d0b9ca47ac965c8638771668b1b24fd0f39bba11ec192d65372bc3e8b3ed2d: Status 404 returned error can't find the container with id b9d0b9ca47ac965c8638771668b1b24fd0f39bba11ec192d65372bc3e8b3ed2d Jul 06 12:48:49.967795 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:49.967754 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bflvj" event={"ID":"9b3622d5-e204-41c2-ba3b-a76597588b25","Type":"ContainerStarted","Data":"b9d0b9ca47ac965c8638771668b1b24fd0f39bba11ec192d65372bc3e8b3ed2d"} Jul 06 12:48:51.977429 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:51.977385 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48wjq" event={"ID":"f82becb4-8b89-4e3e-8687-eb04cc2c6b73","Type":"ContainerStarted","Data":"c505623a847091d4ba7a276b4b3296ae550cc524877ca3566a69c72149da8276"} Jul 06 12:48:51.978932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:51.978909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bflvj" event={"ID":"9b3622d5-e204-41c2-ba3b-a76597588b25","Type":"ContainerStarted","Data":"e7ccf818181b363b424469c4eb36ce006afe0c35efb0ccbc30b00008982365e0"} Jul 06 12:48:51.979056 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:51.978956 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bflvj" event={"ID":"9b3622d5-e204-41c2-ba3b-a76597588b25","Type":"ContainerStarted","Data":"391edfaddab65ca949bf318c68b16b6e2fb9deb129e6c0416c533e65cbd78ef6"} Jul 06 12:48:51.997076 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:51.997032 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-48wjq" podStartSLOduration=18.047654024 podStartE2EDuration="24.997016676s" totalImagesPullingTime="6.949362652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 12:48:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:51.995826859 +0000 UTC m=+132.077267911" watchObservedRunningTime="2026-07-06 12:48:51.997016676 +0000 UTC m=+132.078457703" Jul 06 12:48:52.015075 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:52.015027 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bflvj" podStartSLOduration=129.273511115 podStartE2EDuration="2m11.015011404s" totalImagesPullingTime="1.741500289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:52.013991313 +0000 UTC m=+132.095432360" watchObservedRunningTime="2026-07-06 12:48:52.015011404 +0000 UTC m=+132.096452430" Jul 06 12:48:53.788453 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.788418 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99"] Jul 06 12:48:53.791623 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.791605 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.796545 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.796524 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 12:48:53.797388 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.797369 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 12:48:53.797690 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.797676 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 12:48:53.805388 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.805372 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 12:48:53.820858 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.820837 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99"] Jul 06 12:48:53.844689 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.844663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/98118bcd-ddcc-4aac-9b24-32721312b7a5-klusterlet-config\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.844804 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.844701 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2dkr\" (UniqueName: \"kubernetes.io/projected/98118bcd-ddcc-4aac-9b24-32721312b7a5-kube-api-access-l2dkr\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.844804 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.844726 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98118bcd-ddcc-4aac-9b24-32721312b7a5-tmp\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.862485 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.862466 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc"] Jul 06 12:48:53.865517 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.865501 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:53.872060 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.872038 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 12:48:53.872760 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.872744 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-n7wwd\"" Jul 06 12:48:53.872978 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.872963 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 12:48:53.873241 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.873228 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-qt5jj"] Jul 06 12:48:53.875880 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.875862 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k"] Jul 06 12:48:53.875981 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.875967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:48:53.878692 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.878675 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 12:48:53.878799 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.878715 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:53.878799 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.878758 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 12:48:53.879569 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.879553 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ftkb4\"" Jul 06 12:48:53.882454 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.882436 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 12:48:53.882584 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.882567 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lr2ks\"" Jul 06 12:48:53.892704 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.892686 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc"] Jul 06 12:48:53.897637 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.897618 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-qt5jj"] Jul 06 12:48:53.898995 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.898971 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k"] Jul 06 12:48:53.945956 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.945932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2dkr\" (UniqueName: \"kubernetes.io/projected/98118bcd-ddcc-4aac-9b24-32721312b7a5-kube-api-access-l2dkr\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.946052 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.945959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98118bcd-ddcc-4aac-9b24-32721312b7a5-tmp\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.946052 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.945989 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2mxp\" (UniqueName: \"kubernetes.io/projected/9810c557-1ee9-4ccf-a9b2-e961e8c834e6-kube-api-access-n2mxp\") pod \"downloads-685bfb5f64-qt5jj\" (UID: \"9810c557-1ee9-4ccf-a9b2-e961e8c834e6\") " pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:48:53.946052 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.946007 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d68ba76-1b3b-4f6d-8404-996d7e89a84a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4bnc\" (UID: \"7d68ba76-1b3b-4f6d-8404-996d7e89a84a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:53.946225 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.946068 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d68ba76-1b3b-4f6d-8404-996d7e89a84a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j4bnc\" (UID: \"7d68ba76-1b3b-4f6d-8404-996d7e89a84a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:53.946225 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.946146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/98118bcd-ddcc-4aac-9b24-32721312b7a5-klusterlet-config\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.946225 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.946201 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ff727b66-01c9-4df5-bac4-14b28594babe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fjd6k\" (UID: \"ff727b66-01c9-4df5-bac4-14b28594babe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:53.946379 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.946365 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98118bcd-ddcc-4aac-9b24-32721312b7a5-tmp\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.948705 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.948680 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/98118bcd-ddcc-4aac-9b24-32721312b7a5-klusterlet-config\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:53.966585 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:53.966556 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2dkr\" (UniqueName: \"kubernetes.io/projected/98118bcd-ddcc-4aac-9b24-32721312b7a5-kube-api-access-l2dkr\") pod \"klusterlet-addon-workmgr-678bbd574b-hfn99\" (UID: \"98118bcd-ddcc-4aac-9b24-32721312b7a5\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:54.047496 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.047435 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d68ba76-1b3b-4f6d-8404-996d7e89a84a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j4bnc\" (UID: \"7d68ba76-1b3b-4f6d-8404-996d7e89a84a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:54.047496 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.047485 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ff727b66-01c9-4df5-bac4-14b28594babe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fjd6k\" (UID: \"ff727b66-01c9-4df5-bac4-14b28594babe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:54.047698 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.047525 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2mxp\" (UniqueName: \"kubernetes.io/projected/9810c557-1ee9-4ccf-a9b2-e961e8c834e6-kube-api-access-n2mxp\") pod \"downloads-685bfb5f64-qt5jj\" (UID: \"9810c557-1ee9-4ccf-a9b2-e961e8c834e6\") " pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:48:54.047698 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.047652 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d68ba76-1b3b-4f6d-8404-996d7e89a84a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4bnc\" (UID: \"7d68ba76-1b3b-4f6d-8404-996d7e89a84a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:54.048262 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.048242 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d68ba76-1b3b-4f6d-8404-996d7e89a84a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4bnc\" (UID: \"7d68ba76-1b3b-4f6d-8404-996d7e89a84a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:54.049719 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.049697 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d68ba76-1b3b-4f6d-8404-996d7e89a84a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j4bnc\" (UID: \"7d68ba76-1b3b-4f6d-8404-996d7e89a84a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:54.049855 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.049837 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ff727b66-01c9-4df5-bac4-14b28594babe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-fjd6k\" (UID: \"ff727b66-01c9-4df5-bac4-14b28594babe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:54.077551 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.077528 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2mxp\" (UniqueName: \"kubernetes.io/projected/9810c557-1ee9-4ccf-a9b2-e961e8c834e6-kube-api-access-n2mxp\") pod \"downloads-685bfb5f64-qt5jj\" (UID: \"9810c557-1ee9-4ccf-a9b2-e961e8c834e6\") " pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:48:54.100946 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.100911 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:54.174437 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.174411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" Jul 06 12:48:54.185834 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.185804 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:48:54.189951 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.189843 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:54.250963 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.250912 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99"] Jul 06 12:48:54.256407 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:54.256371 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98118bcd_ddcc_4aac_9b24_32721312b7a5.slice/crio-7e0497b359baa5e8af741b25e122235a5af6c12c23d485cc2818868b47cb3902 WatchSource:0}: Error finding container 7e0497b359baa5e8af741b25e122235a5af6c12c23d485cc2818868b47cb3902: Status 404 returned error can't find the container with id 7e0497b359baa5e8af741b25e122235a5af6c12c23d485cc2818868b47cb3902 Jul 06 12:48:54.343848 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.343776 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc"] Jul 06 12:48:54.347187 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:54.347142 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d68ba76_1b3b_4f6d_8404_996d7e89a84a.slice/crio-603717b0ccba6bf1ab3d43d9960021ab65934284660fb3e402e6d328457b4643 WatchSource:0}: Error finding container 603717b0ccba6bf1ab3d43d9960021ab65934284660fb3e402e6d328457b4643: Status 404 returned error can't find the container with id 603717b0ccba6bf1ab3d43d9960021ab65934284660fb3e402e6d328457b4643 Jul 06 12:48:54.556111 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.556090 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k"] Jul 06 12:48:54.557953 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:54.557920 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff727b66_01c9_4df5_bac4_14b28594babe.slice/crio-bdaeebffb68df283b48b4da70eb13085967a46ec3b6748633e2274d0c27b6d04 WatchSource:0}: Error finding container bdaeebffb68df283b48b4da70eb13085967a46ec3b6748633e2274d0c27b6d04: Status 404 returned error can't find the container with id bdaeebffb68df283b48b4da70eb13085967a46ec3b6748633e2274d0c27b6d04 Jul 06 12:48:54.558431 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.558411 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-qt5jj"] Jul 06 12:48:54.561754 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:54.561734 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9810c557_1ee9_4ccf_a9b2_e961e8c834e6.slice/crio-d21c7acc3323c94bc38f956f23c3f68f308a7043587270f6bde28b4d9592f01b WatchSource:0}: Error finding container d21c7acc3323c94bc38f956f23c3f68f308a7043587270f6bde28b4d9592f01b: Status 404 returned error can't find the container with id d21c7acc3323c94bc38f956f23c3f68f308a7043587270f6bde28b4d9592f01b Jul 06 12:48:54.991476 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.991442 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" event={"ID":"ff727b66-01c9-4df5-bac4-14b28594babe","Type":"ContainerStarted","Data":"bdaeebffb68df283b48b4da70eb13085967a46ec3b6748633e2274d0c27b6d04"} Jul 06 12:48:54.992770 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.992745 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-qt5jj" event={"ID":"9810c557-1ee9-4ccf-a9b2-e961e8c834e6","Type":"ContainerStarted","Data":"d21c7acc3323c94bc38f956f23c3f68f308a7043587270f6bde28b4d9592f01b"} Jul 06 12:48:54.993958 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.993930 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" event={"ID":"7d68ba76-1b3b-4f6d-8404-996d7e89a84a","Type":"ContainerStarted","Data":"603717b0ccba6bf1ab3d43d9960021ab65934284660fb3e402e6d328457b4643"} Jul 06 12:48:54.995091 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:54.995066 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" event={"ID":"98118bcd-ddcc-4aac-9b24-32721312b7a5","Type":"ContainerStarted","Data":"7e0497b359baa5e8af741b25e122235a5af6c12c23d485cc2818868b47cb3902"} Jul 06 12:48:56.001753 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:56.001527 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" event={"ID":"7d68ba76-1b3b-4f6d-8404-996d7e89a84a","Type":"ContainerStarted","Data":"e032eb92050b2801e63b9d4febd6a9533cc37509d03a06092bc2274abba36f17"} Jul 06 12:48:56.021850 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:56.021795 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4bnc" podStartSLOduration=1.795045988 podStartE2EDuration="3.021775353s" totalImagesPullingTime="1.226729365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:56.021495616 +0000 UTC m=+136.102936669" watchObservedRunningTime="2026-07-06 12:48:56.021775353 +0000 UTC m=+136.103216375" Jul 06 12:48:58.577135 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.577107 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-768fc9c5db-lrz4s"] Jul 06 12:48:58.580132 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.580111 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.588201 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.588102 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 12:48:58.589528 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.589509 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 12:48:58.589754 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.589733 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jwqgr\"" Jul 06 12:48:58.589851 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.589832 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 12:48:58.589960 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.589732 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 12:48:58.590033 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.590022 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 12:48:58.597877 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.597854 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768fc9c5db-lrz4s"] Jul 06 12:48:58.690839 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.690804 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg62q\" (UniqueName: \"kubernetes.io/projected/7f6103e4-008e-4d58-abbb-473bd3da52f5-kube-api-access-vg62q\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.691002 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.690853 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-config\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.691002 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.690963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-oauth-serving-cert\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.691119 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.691010 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-oauth-config\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.691119 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.691089 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-service-ca\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.691248 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.691131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-serving-cert\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.792334 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.792256 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-config\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.792334 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.792328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-oauth-serving-cert\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.792552 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.792369 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-oauth-config\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.792552 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.792408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-service-ca\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.792552 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.792454 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-serving-cert\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.792552 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.792496 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg62q\" (UniqueName: \"kubernetes.io/projected/7f6103e4-008e-4d58-abbb-473bd3da52f5-kube-api-access-vg62q\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.793118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.793059 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-oauth-serving-cert\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.793278 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.793140 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-service-ca\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.793278 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.793161 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-config\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.795115 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.795082 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-oauth-config\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.795286 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.795269 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-serving-cert\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.802373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.802350 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg62q\" (UniqueName: \"kubernetes.io/projected/7f6103e4-008e-4d58-abbb-473bd3da52f5-kube-api-access-vg62q\") pod \"console-768fc9c5db-lrz4s\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:58.906792 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:58.906765 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:48:59.024091 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.024050 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" event={"ID":"98118bcd-ddcc-4aac-9b24-32721312b7a5","Type":"ContainerStarted","Data":"f9b0955d8c6f7e73a66a93b9c2f64819d289b2f38263cd05bc7eea9a8e85b5c4"} Jul 06 12:48:59.024261 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.024184 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:59.025933 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.025875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" Jul 06 12:48:59.026072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.026037 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" event={"ID":"ff727b66-01c9-4df5-bac4-14b28594babe","Type":"ContainerStarted","Data":"bed5dfdf2c49f0057189bef61bbd6f300ebb2e5933dd708c262b1c12504bf96a"} Jul 06 12:48:59.026341 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.026295 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:59.031265 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.031230 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" Jul 06 12:48:59.050396 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.050318 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-678bbd574b-hfn99" podStartSLOduration=1.758397626 podStartE2EDuration="6.050302167s" totalImagesPullingTime="4.291904541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:59.048522977 +0000 UTC m=+139.129964005" watchObservedRunningTime="2026-07-06 12:48:59.050302167 +0000 UTC m=+139.131743194" Jul 06 12:48:59.079123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.079098 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768fc9c5db-lrz4s"] Jul 06 12:48:59.082363 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:48:59.082340 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6103e4_008e_4d58_abbb_473bd3da52f5.slice/crio-c4653d8b6d9ed85072f09737b9f97031adc4dcb1cf26cd357da85ac4fe8b341c WatchSource:0}: Error finding container c4653d8b6d9ed85072f09737b9f97031adc4dcb1cf26cd357da85ac4fe8b341c: Status 404 returned error can't find the container with id c4653d8b6d9ed85072f09737b9f97031adc4dcb1cf26cd357da85ac4fe8b341c Jul 06 12:48:59.084689 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:48:59.084639 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-fjd6k" podStartSLOduration=2.164748342 podStartE2EDuration="6.084623294s" totalImagesPullingTime="3.919874952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:48:59.082760509 +0000 UTC m=+139.164201554" watchObservedRunningTime="2026-07-06 12:48:59.084623294 +0000 UTC m=+139.166064313" Jul 06 12:49:00.031706 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:00.031668 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fc9c5db-lrz4s" event={"ID":"7f6103e4-008e-4d58-abbb-473bd3da52f5","Type":"ContainerStarted","Data":"c4653d8b6d9ed85072f09737b9f97031adc4dcb1cf26cd357da85ac4fe8b341c"} Jul 06 12:49:03.044367 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:03.044335 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fc9c5db-lrz4s" event={"ID":"7f6103e4-008e-4d58-abbb-473bd3da52f5","Type":"ContainerStarted","Data":"954dec5d13a3d0ca18e55caf536660d3eabbea0bdfff3414c8a1233524ac2cf4"} Jul 06 12:49:03.064806 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:03.064755 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-768fc9c5db-lrz4s" podStartSLOduration=1.808535572 podStartE2EDuration="5.064740203s" totalImagesPullingTime="3.256204631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:03.064124611 +0000 UTC m=+143.145565639" watchObservedRunningTime="2026-07-06 12:49:03.064740203 +0000 UTC m=+143.146181221" Jul 06 12:49:04.050982 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.050944 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp"] Jul 06 12:49:04.054474 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.054454 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.057807 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.057782 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 12:49:04.058961 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.058936 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 12:49:04.059160 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.059148 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-m5zdx\"" Jul 06 12:49:04.059228 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.059194 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 12:49:04.064568 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.064547 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-44xk2"] Jul 06 12:49:04.067518 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.067502 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.071025 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.071009 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 12:49:04.071831 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.071805 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 12:49:04.071917 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.071901 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dp8jt\"" Jul 06 12:49:04.071954 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.071925 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 12:49:04.081931 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.081896 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp"] Jul 06 12:49:04.143037 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.143208 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rld9r\" (UniqueName: \"kubernetes.io/projected/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-kube-api-access-rld9r\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.143208 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-root\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143208 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143132 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-tls\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143208 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.143421 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e6d8030-618d-448b-ac5c-050744d195a2-metrics-client-ca\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143421 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143336 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-wtmp\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143421 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143527 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-textfile\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143527 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzws\" (UniqueName: \"kubernetes.io/projected/9e6d8030-618d-448b-ac5c-050744d195a2-kube-api-access-mtzws\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143527 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143505 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.143671 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143546 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.143671 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.143592 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-sys\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.244704 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244670 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rld9r\" (UniqueName: \"kubernetes.io/projected/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-kube-api-access-rld9r\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.244887 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-root\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.244887 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244785 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-root\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.244887 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244839 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-tls\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.244887 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244861 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.245123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e6d8030-618d-448b-ac5c-050744d195a2-metrics-client-ca\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.244914 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-wtmp\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245056 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-wtmp\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245365 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245128 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-textfile\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245365 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzws\" (UniqueName: \"kubernetes.io/projected/9e6d8030-618d-448b-ac5c-050744d195a2-kube-api-access-mtzws\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245365 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245220 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245365 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.245365 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-sys\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245365 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.245757 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245734 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-textfile\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245837 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245733 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.245837 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245789 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e6d8030-618d-448b-ac5c-050744d195a2-sys\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.245949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.245870 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e6d8030-618d-448b-ac5c-050744d195a2-metrics-client-ca\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.246237 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.246206 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-accelerators-collector-config\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.247863 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.247810 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-tls\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.248153 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.248129 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e6d8030-618d-448b-ac5c-050744d195a2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.248419 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.248395 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.248499 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.248461 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.260722 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.260692 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzws\" (UniqueName: \"kubernetes.io/projected/9e6d8030-618d-448b-ac5c-050744d195a2-kube-api-access-mtzws\") pod \"node-exporter-44xk2\" (UID: \"9e6d8030-618d-448b-ac5c-050744d195a2\") " pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.262614 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.262587 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rld9r\" (UniqueName: \"kubernetes.io/projected/693f53d4-7f20-47dd-b4d4-48e57e5eb60f-kube-api-access-rld9r\") pod \"openshift-state-metrics-54cf96dfcf-wgmgp\" (UID: \"693f53d4-7f20-47dd-b4d4-48e57e5eb60f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.363702 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.363623 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" Jul 06 12:49:04.375987 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.375963 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44xk2" Jul 06 12:49:04.386931 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:04.386900 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6d8030_618d_448b_ac5c_050744d195a2.slice/crio-c508d37643d1c3277cfb21b54db0856bc15768b35c2a88737476ea5305d947e2 WatchSource:0}: Error finding container c508d37643d1c3277cfb21b54db0856bc15768b35c2a88737476ea5305d947e2: Status 404 returned error can't find the container with id c508d37643d1c3277cfb21b54db0856bc15768b35c2a88737476ea5305d947e2 Jul 06 12:49:04.513507 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:04.513461 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp"] Jul 06 12:49:04.517777 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:04.517743 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693f53d4_7f20_47dd_b4d4_48e57e5eb60f.slice/crio-33f79588e72c652a5bb8cfed9e3d663bdcaa2d78c5300a418e116d415481c611 WatchSource:0}: Error finding container 33f79588e72c652a5bb8cfed9e3d663bdcaa2d78c5300a418e116d415481c611: Status 404 returned error can't find the container with id 33f79588e72c652a5bb8cfed9e3d663bdcaa2d78c5300a418e116d415481c611 Jul 06 12:49:05.056217 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.056155 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" event={"ID":"693f53d4-7f20-47dd-b4d4-48e57e5eb60f","Type":"ContainerStarted","Data":"28a7a7e4dd2b73acab0571a959a0ae72e8a262be54e7b407085e3213301f89ea"} Jul 06 12:49:05.056691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.056259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" event={"ID":"693f53d4-7f20-47dd-b4d4-48e57e5eb60f","Type":"ContainerStarted","Data":"5b081faea80de3f0022b6d3cf5ad4617af692cbc882ea21ff50ad98f7b57e66d"} Jul 06 12:49:05.056691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.056295 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" event={"ID":"693f53d4-7f20-47dd-b4d4-48e57e5eb60f","Type":"ContainerStarted","Data":"33f79588e72c652a5bb8cfed9e3d663bdcaa2d78c5300a418e116d415481c611"} Jul 06 12:49:05.058072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.058033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44xk2" event={"ID":"9e6d8030-618d-448b-ac5c-050744d195a2","Type":"ContainerStarted","Data":"c508d37643d1c3277cfb21b54db0856bc15768b35c2a88737476ea5305d947e2"} Jul 06 12:49:05.143121 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.143026 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:49:05.147322 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.147297 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.150403 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.150374 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 12:49:05.155143 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.154958 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 12:49:05.155564 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.155352 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.155873 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.155928 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.156268 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.156306 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.156561 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.156795 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-45777\"" Jul 06 12:49:05.156970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.156807 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 12:49:05.251324 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.251223 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.253964 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254018 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-out\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254109 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-web-config\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254261 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254378 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254429 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254459 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fdkq\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-kube-api-access-6fdkq\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.254921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.254493 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355111 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355016 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355111 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355072 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355111 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355103 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fdkq\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-kube-api-access-6fdkq\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-out\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355367 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355400 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.355838 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.355412 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-web-config\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.357712 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:49:05.356137 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle podName:1f143d1f-8afa-4db2-96ce-33e4fc6af97a nodeName:}" failed. No retries permitted until 2026-07-06 12:49:05.856114311 +0000 UTC m=+145.937555328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a") : configmap references non-existent config key: ca-bundle.crt Jul 06 12:49:05.357712 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.357005 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.358907 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:49:05.358480 2559 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 12:49:05.358907 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:49:05.358553 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls podName:1f143d1f-8afa-4db2-96ce-33e4fc6af97a nodeName:}" failed. No retries permitted until 2026-07-06 12:49:05.85853522 +0000 UTC m=+145.939976239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a") : secret "alertmanager-main-tls" not found Jul 06 12:49:05.358907 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.358900 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.364238 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.363845 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-out\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.364238 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.364015 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-web-config\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.364238 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.364200 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.364835 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.364790 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.366250 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.366198 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.366607 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.366584 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.366948 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.366928 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-volume\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.368478 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.368454 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.368585 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.368477 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fdkq\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-kube-api-access-6fdkq\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.859922 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.859880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.860103 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.860020 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.861425 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.860996 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:05.864916 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:05.864877 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:06.059156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:06.059104 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:49:06.065928 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:06.065799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44xk2" event={"ID":"9e6d8030-618d-448b-ac5c-050744d195a2","Type":"ContainerStarted","Data":"37930921c410241ef41c9d34208af3c21ffbc82cf53dee5781c6cefd57e9333f"} Jul 06 12:49:06.071156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:06.071084 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" event={"ID":"693f53d4-7f20-47dd-b4d4-48e57e5eb60f","Type":"ContainerStarted","Data":"1adbf486af0512f8efee8702e262991d9116dcfcfe0418401582cb9b60bcf0ed"} Jul 06 12:49:06.147776 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:06.147607 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-wgmgp" podStartSLOduration=0.835948109 podStartE2EDuration="2.147588044s" totalImagesPullingTime="1.311639935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:06.14698862 +0000 UTC m=+146.228429648" watchObservedRunningTime="2026-07-06 12:49:06.147588044 +0000 UTC m=+146.229029072" Jul 06 12:49:06.241711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:06.241684 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:49:06.244384 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:06.244356 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f143d1f_8afa_4db2_96ce_33e4fc6af97a.slice/crio-e31fd5da8787f3a4f1e021fde7935c04552a904395c481848eea7622f9c98dfa WatchSource:0}: Error finding container e31fd5da8787f3a4f1e021fde7935c04552a904395c481848eea7622f9c98dfa: Status 404 returned error can't find the container with id e31fd5da8787f3a4f1e021fde7935c04552a904395c481848eea7622f9c98dfa Jul 06 12:49:06.999585 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:06.999553 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dbd46688b-gxz4k"] Jul 06 12:49:07.002836 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.002811 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.070792 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.070756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-service-ca\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.071272 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.070831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-oauth-config\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.071272 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.070915 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-config\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.071272 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.070941 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-trusted-ca-bundle\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.071272 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.070964 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mlj5\" (UniqueName: \"kubernetes.io/projected/14045b63-d865-43f6-94bb-ed4eddd6fea7-kube-api-access-6mlj5\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.071272 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.071042 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-serving-cert\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.071272 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.071182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-oauth-serving-cert\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.077124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.077090 2559 generic.go:356] "Generic (PLEG): container finished" podID="9e6d8030-618d-448b-ac5c-050744d195a2" containerID="37930921c410241ef41c9d34208af3c21ffbc82cf53dee5781c6cefd57e9333f" exitCode=0 Jul 06 12:49:07.077311 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.077217 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44xk2" event={"ID":"9e6d8030-618d-448b-ac5c-050744d195a2","Type":"ContainerDied","Data":"37930921c410241ef41c9d34208af3c21ffbc82cf53dee5781c6cefd57e9333f"} Jul 06 12:49:07.079103 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.079076 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"e31fd5da8787f3a4f1e021fde7935c04552a904395c481848eea7622f9c98dfa"} Jul 06 12:49:07.087552 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.087497 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbd46688b-gxz4k"] Jul 06 12:49:07.099837 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.099810 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.172689 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-oauth-serving-cert\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.172800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-service-ca\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.172871 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-oauth-config\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.172913 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-config\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.172937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-trusted-ca-bundle\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.172959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mlj5\" (UniqueName: \"kubernetes.io/projected/14045b63-d865-43f6-94bb-ed4eddd6fea7-kube-api-access-6mlj5\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.173045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-serving-cert\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.174095 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.173761 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-oauth-serving-cert\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.175710 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.174764 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-service-ca\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.175710 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.175357 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-config\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.176525 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.176216 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-trusted-ca-bundle\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.178765 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.178727 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-serving-cert\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.182713 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.182669 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-oauth-config\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.195053 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.194998 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mlj5\" (UniqueName: \"kubernetes.io/projected/14045b63-d865-43f6-94bb-ed4eddd6fea7-kube-api-access-6mlj5\") pod \"console-7dbd46688b-gxz4k\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.316194 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.316097 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:07.751830 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:07.751800 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dbd46688b-gxz4k"] Jul 06 12:49:07.788203 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:07.788150 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14045b63_d865_43f6_94bb_ed4eddd6fea7.slice/crio-c77b6d8e83196f151adfd26b94691cf985ce07f2d377e74bb3803c6588cde424 WatchSource:0}: Error finding container c77b6d8e83196f151adfd26b94691cf985ce07f2d377e74bb3803c6588cde424: Status 404 returned error can't find the container with id c77b6d8e83196f151adfd26b94691cf985ce07f2d377e74bb3803c6588cde424 Jul 06 12:49:08.086428 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.086389 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44xk2" event={"ID":"9e6d8030-618d-448b-ac5c-050744d195a2","Type":"ContainerStarted","Data":"1bf270a75168df6ceb8c3e86bad5a488cb62693e60699bf6c82c0029408b8f2f"} Jul 06 12:49:08.086922 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.086457 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44xk2" event={"ID":"9e6d8030-618d-448b-ac5c-050744d195a2","Type":"ContainerStarted","Data":"7658f9f1206dc3340ba7c6431b917ab824b292f45b74e3bffc4ec401c63342ad"} Jul 06 12:49:08.088031 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.088005 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4" exitCode=0 Jul 06 12:49:08.088159 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.088041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4"} Jul 06 12:49:08.089886 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.089804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd46688b-gxz4k" event={"ID":"14045b63-d865-43f6-94bb-ed4eddd6fea7","Type":"ContainerStarted","Data":"ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5"} Jul 06 12:49:08.089886 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.089843 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd46688b-gxz4k" event={"ID":"14045b63-d865-43f6-94bb-ed4eddd6fea7","Type":"ContainerStarted","Data":"c77b6d8e83196f151adfd26b94691cf985ce07f2d377e74bb3803c6588cde424"} Jul 06 12:49:08.132569 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.132507 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-44xk2" podStartSLOduration=2.463012329 podStartE2EDuration="4.132484563s" totalImagesPullingTime="1.576186965s" totalInitContainerRuntime="93.285269ms" isStatefulPod=true podCreationTimestamp="2026-07-06 12:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:08.132185073 +0000 UTC m=+148.213626095" watchObservedRunningTime="2026-07-06 12:49:08.132484563 +0000 UTC m=+148.213925592" Jul 06 12:49:08.220639 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.220587 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dbd46688b-gxz4k" podStartSLOduration=2.220569073 podStartE2EDuration="2.220569073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:08.218698274 +0000 UTC m=+148.300139329" watchObservedRunningTime="2026-07-06 12:49:08.220569073 +0000 UTC m=+148.302010100" Jul 06 12:49:08.626343 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.626307 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-649c995458-8xxxk"] Jul 06 12:49:08.630673 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.630653 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.635471 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.635443 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 12:49:08.635788 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.635757 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5psrt\"" Jul 06 12:49:08.636507 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.636386 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7ipvdr3r8n1le\"" Jul 06 12:49:08.636610 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.636543 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 12:49:08.636610 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.636556 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 12:49:08.636610 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.636565 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 12:49:08.666652 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.666622 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-649c995458-8xxxk"] Jul 06 12:49:08.687263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687228 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-client-ca-bundle\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.687263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687267 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-secret-metrics-server-tls\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.687495 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-secret-metrics-server-client-certs\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.687495 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687470 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2d620d2f-94b4-4090-85ef-4ff6c237b25b-metrics-server-audit-profiles\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.687655 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687499 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w7w9\" (UniqueName: \"kubernetes.io/projected/2d620d2f-94b4-4090-85ef-4ff6c237b25b-kube-api-access-2w7w9\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.687655 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687539 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d620d2f-94b4-4090-85ef-4ff6c237b25b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.687655 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.687640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2d620d2f-94b4-4090-85ef-4ff6c237b25b-audit-log\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789044 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789007 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-client-ca-bundle\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-secret-metrics-server-tls\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789122 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-secret-metrics-server-client-certs\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789197 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2d620d2f-94b4-4090-85ef-4ff6c237b25b-metrics-server-audit-profiles\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789263 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789234 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w7w9\" (UniqueName: \"kubernetes.io/projected/2d620d2f-94b4-4090-85ef-4ff6c237b25b-kube-api-access-2w7w9\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789499 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d620d2f-94b4-4090-85ef-4ff6c237b25b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789499 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789345 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2d620d2f-94b4-4090-85ef-4ff6c237b25b-audit-log\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.789715 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.789688 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2d620d2f-94b4-4090-85ef-4ff6c237b25b-audit-log\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.790266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.790218 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d620d2f-94b4-4090-85ef-4ff6c237b25b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.790411 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.790380 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2d620d2f-94b4-4090-85ef-4ff6c237b25b-metrics-server-audit-profiles\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.792072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.791978 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-secret-metrics-server-tls\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.792072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.792042 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-client-ca-bundle\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.792295 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.792270 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2d620d2f-94b4-4090-85ef-4ff6c237b25b-secret-metrics-server-client-certs\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.799148 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.799127 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w7w9\" (UniqueName: \"kubernetes.io/projected/2d620d2f-94b4-4090-85ef-4ff6c237b25b-kube-api-access-2w7w9\") pod \"metrics-server-649c995458-8xxxk\" (UID: \"2d620d2f-94b4-4090-85ef-4ff6c237b25b\") " pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.907741 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.907649 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:49:08.907741 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.907718 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:49:08.913656 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.913626 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:49:08.942602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.942563 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:08.987972 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:08.987939 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768fc9c5db-lrz4s"] Jul 06 12:49:09.099676 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:09.099643 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:49:10.820458 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.820420 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:49:10.825646 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.825622 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.849958 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.849925 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 12:49:10.850138 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.849938 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 12:49:10.850297 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.850279 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 12:49:10.850680 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.850558 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 12:49:10.850680 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.850594 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 12:49:10.851031 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.851013 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-dcwpq\"" Jul 06 12:49:10.851269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.851252 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 12:49:10.862241 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.862214 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 12:49:10.862427 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.862409 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 12:49:10.862521 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.862506 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 12:49:10.862586 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.862567 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 12:49:10.862801 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.862786 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7icfv464d1s36\"" Jul 06 12:49:10.869271 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.869215 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 12:49:10.877822 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.877796 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 12:49:10.878345 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.878324 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 12:49:10.897057 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.897010 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 12:49:10.913339 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913306 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913524 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913353 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-config\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913524 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913390 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-config-out\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913524 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913418 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913684 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913684 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913584 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913684 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913606 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913684 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913675 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-web-config\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913808 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913833 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913851 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.913873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913872 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9kh8\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-kube-api-access-m9kh8\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.914278 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913895 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.914278 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913929 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.914278 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.913982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.914278 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.914015 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:10.927229 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:10.927195 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:49:11.014759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.014725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-web-config\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.014947 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.014780 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015002 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.014979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015050 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015192 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015152 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015258 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015217 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015258 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015251 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015366 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015285 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9kh8\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-kube-api-access-m9kh8\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015366 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015324 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015366 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015356 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015513 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015513 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015513 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015466 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015513 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015500 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-config\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-config-out\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015601 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015646 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015658 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.015711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015688 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.016050 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.016050 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.015965 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.016290 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.016262 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.016385 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.016344 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.017381 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.017330 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.018025 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.017979 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.018615 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.018587 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.018615 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.018606 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.020110 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.020063 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-web-config\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.020235 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.020145 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.020310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.020235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.020664 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.020627 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-config\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.021062 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.021015 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.021279 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.021259 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.021374 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.021259 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.021763 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.021741 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-config-out\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.022123 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.022099 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.022218 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.022200 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.049404 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.049373 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9kh8\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-kube-api-access-m9kh8\") pod \"prometheus-k8s-0\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:11.141180 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:11.137465 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:15.862915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:15.862891 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:49:15.864463 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:15.864440 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-649c995458-8xxxk"] Jul 06 12:49:15.888185 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:15.888122 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fafa7a5_b356_417a_8d1e_251cf404febe.slice/crio-e1612ef31a51f82039fdd00dc2fb7a394357cea7a099e61d4a8f1fe8560dae74 WatchSource:0}: Error finding container e1612ef31a51f82039fdd00dc2fb7a394357cea7a099e61d4a8f1fe8560dae74: Status 404 returned error can't find the container with id e1612ef31a51f82039fdd00dc2fb7a394357cea7a099e61d4a8f1fe8560dae74 Jul 06 12:49:15.889015 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:15.888994 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d620d2f_94b4_4090_85ef_4ff6c237b25b.slice/crio-506e61b5d9e220c174b2e1010c821c2d0cbf359c9cd0f7be43c307a9e9a715af WatchSource:0}: Error finding container 506e61b5d9e220c174b2e1010c821c2d0cbf359c9cd0f7be43c307a9e9a715af: Status 404 returned error can't find the container with id 506e61b5d9e220c174b2e1010c821c2d0cbf359c9cd0f7be43c307a9e9a715af Jul 06 12:49:16.128468 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.128419 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" exitCode=0 Jul 06 12:49:16.128661 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.128481 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} Jul 06 12:49:16.128661 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.128551 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"e1612ef31a51f82039fdd00dc2fb7a394357cea7a099e61d4a8f1fe8560dae74"} Jul 06 12:49:16.129974 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.129945 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" event={"ID":"2d620d2f-94b4-4090-85ef-4ff6c237b25b","Type":"ContainerStarted","Data":"506e61b5d9e220c174b2e1010c821c2d0cbf359c9cd0f7be43c307a9e9a715af"} Jul 06 12:49:16.132139 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.132076 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-qt5jj" event={"ID":"9810c557-1ee9-4ccf-a9b2-e961e8c834e6","Type":"ContainerStarted","Data":"2effe3ac74f1674dc0ed4607bf4ba14e0c528e38e828d0f0b9d31463575f42e0"} Jul 06 12:49:16.132328 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.132307 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:49:16.134025 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.134002 2559 patch_prober.go:28] interesting pod/downloads-685bfb5f64-qt5jj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" start-of-body= Jul 06 12:49:16.134119 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.134048 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-qt5jj" podUID="9810c557-1ee9-4ccf-a9b2-e961e8c834e6" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" Jul 06 12:49:16.195923 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:16.195844 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78ccf75cfd-xps57"] Jul 06 12:49:16.196255 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:49:16.196233 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" podUID="16edb0eb-f728-4318-9245-e64587cb9253" Jul 06 12:49:17.139652 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.139612 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:49:17.155633 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.155604 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-qt5jj" Jul 06 12:49:17.167571 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.167425 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:49:17.205266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.205178 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-qt5jj" podStartSLOduration=2.825461844 podStartE2EDuration="24.205140113s" totalImagesPullingTime="21.379678269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:16.30702593 +0000 UTC m=+156.388466958" watchObservedRunningTime="2026-07-06 12:49:17.205140113 +0000 UTC m=+157.286581142" Jul 06 12:49:17.276567 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276063 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-bound-sa-token\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276567 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276135 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmglj\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-kube-api-access-hmglj\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276567 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276191 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-registry-certificates\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276567 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276252 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-trusted-ca\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276912 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276589 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-installation-pull-secrets\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276912 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276703 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16edb0eb-f728-4318-9245-e64587cb9253-ca-trust-extracted\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276912 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276748 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-image-registry-private-configuration\") pod \"16edb0eb-f728-4318-9245-e64587cb9253\" (UID: \"16edb0eb-f728-4318-9245-e64587cb9253\") " Jul 06 12:49:17.276912 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.276873 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:17.277262 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.277144 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-trusted-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:17.277493 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.277458 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:17.277636 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.277593 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16edb0eb-f728-4318-9245-e64587cb9253-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:49:17.289689 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.289628 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-kube-api-access-hmglj" (OuterVolumeSpecName: "kube-api-access-hmglj") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "kube-api-access-hmglj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:49:17.290691 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.290651 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:49:17.290970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.290941 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:49:17.291256 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.291224 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "16edb0eb-f728-4318-9245-e64587cb9253" (UID: "16edb0eb-f728-4318-9245-e64587cb9253"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:49:17.317536 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.317278 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:17.317723 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.317660 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:17.324194 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.323960 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:17.377913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.377787 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmglj\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-kube-api-access-hmglj\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:17.377913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.377820 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/16edb0eb-f728-4318-9245-e64587cb9253-registry-certificates\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:17.377913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.377836 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-installation-pull-secrets\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:17.377913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.377850 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/16edb0eb-f728-4318-9245-e64587cb9253-ca-trust-extracted\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:17.377913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.377864 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/16edb0eb-f728-4318-9245-e64587cb9253-image-registry-private-configuration\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:17.377913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:17.377879 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-bound-sa-token\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:18.147876 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:18.147559 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ccf75cfd-xps57" Jul 06 12:49:18.159394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:18.159338 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:18.302779 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:18.302738 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78ccf75cfd-xps57"] Jul 06 12:49:18.312770 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:18.312735 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78ccf75cfd-xps57"] Jul 06 12:49:18.390380 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:18.390341 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/16edb0eb-f728-4318-9245-e64587cb9253-registry-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:18.417242 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:18.417191 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16edb0eb-f728-4318-9245-e64587cb9253" path="/var/lib/kubelet/pods/16edb0eb-f728-4318-9245-e64587cb9253/volumes" Jul 06 12:49:19.156026 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:19.155860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" event={"ID":"2d620d2f-94b4-4090-85ef-4ff6c237b25b","Type":"ContainerStarted","Data":"3353f33cab251e7f1ffc770a5008b9d6ff39a2482c65c603afa7be974716078f"} Jul 06 12:49:19.159127 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:19.159087 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08"} Jul 06 12:49:19.590850 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:19.590787 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" podStartSLOduration=8.709509197 podStartE2EDuration="11.590734316s" totalImagesPullingTime="2.881225119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:19.199912474 +0000 UTC m=+159.281353519" watchObservedRunningTime="2026-07-06 12:49:19.590734316 +0000 UTC m=+159.672175345" Jul 06 12:49:19.592924 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:19.592851 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbd46688b-gxz4k"] Jul 06 12:49:20.762015 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:49:20.761971 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-stdbh" podUID="6990e881-e5ad-4208-9548-3ccabbff5f89" Jul 06 12:49:21.183380 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.183288 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} Jul 06 12:49:21.183633 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.183403 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} Jul 06 12:49:21.194018 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.190894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-stdbh" Jul 06 12:49:21.194018 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.191293 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416"} Jul 06 12:49:21.194018 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.191363 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8"} Jul 06 12:49:21.194018 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.191406 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0"} Jul 06 12:49:21.194018 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:21.191445 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15"} Jul 06 12:49:24.212326 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:24.212282 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerStarted","Data":"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2"} Jul 06 12:49:24.216221 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:24.216191 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} Jul 06 12:49:24.259971 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:24.259907 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.690580327 podStartE2EDuration="19.259882466s" totalImagesPullingTime="14.553292722s" totalInitContainerRuntime="16.009417ms" isStatefulPod=false podCreationTimestamp="2026-07-06 12:49:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:24.258206623 +0000 UTC m=+164.339647651" watchObservedRunningTime="2026-07-06 12:49:24.259882466 +0000 UTC m=+164.341323494" Jul 06 12:49:25.225239 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.225197 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} Jul 06 12:49:25.225843 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.225266 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} Jul 06 12:49:25.225843 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.225292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerStarted","Data":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} Jul 06 12:49:25.264218 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.264127 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.718748376 podStartE2EDuration="15.264102524s" totalImagesPullingTime="7.516173881s" totalInitContainerRuntime="29.180267ms" isStatefulPod=false podCreationTimestamp="2026-07-06 12:49:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:25.261691152 +0000 UTC m=+165.343132204" watchObservedRunningTime="2026-07-06 12:49:25.264102524 +0000 UTC m=+165.345543552" Jul 06 12:49:25.672789 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.672748 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:49:25.675267 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.675242 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6990e881-e5ad-4208-9548-3ccabbff5f89-metrics-tls\") pod \"dns-default-stdbh\" (UID: \"6990e881-e5ad-4208-9548-3ccabbff5f89\") " pod="openshift-dns/dns-default-stdbh" Jul 06 12:49:25.695432 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.695408 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9njlh\"" Jul 06 12:49:25.702863 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.702840 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-stdbh" Jul 06 12:49:25.884897 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:25.884860 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-stdbh"] Jul 06 12:49:25.888612 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:49:25.888574 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6990e881_e5ad_4208_9548_3ccabbff5f89.slice/crio-b34967c286fe55f6f8fe455dbe5f9f1a82677105932a67e05e52561bf1d1aa1f WatchSource:0}: Error finding container b34967c286fe55f6f8fe455dbe5f9f1a82677105932a67e05e52561bf1d1aa1f: Status 404 returned error can't find the container with id b34967c286fe55f6f8fe455dbe5f9f1a82677105932a67e05e52561bf1d1aa1f Jul 06 12:49:26.138207 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:26.138102 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:49:26.233817 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:26.233452 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-stdbh" event={"ID":"6990e881-e5ad-4208-9548-3ccabbff5f89","Type":"ContainerStarted","Data":"b34967c286fe55f6f8fe455dbe5f9f1a82677105932a67e05e52561bf1d1aa1f"} Jul 06 12:49:28.943284 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:28.943249 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:28.943694 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:28.943315 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:29.252051 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:29.251949 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-stdbh" event={"ID":"6990e881-e5ad-4208-9548-3ccabbff5f89","Type":"ContainerStarted","Data":"49af779d38eda8af7bcbdceabefab4b16b666d837ab810daadfad7cccbe5ccec"} Jul 06 12:49:29.252051 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:29.252024 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-stdbh" event={"ID":"6990e881-e5ad-4208-9548-3ccabbff5f89","Type":"ContainerStarted","Data":"fcf4746f569bd7c5a3134bcb4669cfd24591b3f050c0ca62559c3d62d1a73695"} Jul 06 12:49:29.252298 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:29.252058 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-stdbh" Jul 06 12:49:29.292526 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:29.292475 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-stdbh" podStartSLOduration=129.838052726 podStartE2EDuration="2m12.292457586s" totalImagesPullingTime="2.45440486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:49:29.291715861 +0000 UTC m=+169.373156900" watchObservedRunningTime="2026-07-06 12:49:29.292457586 +0000 UTC m=+169.373898613" Jul 06 12:49:35.122377 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.122262 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-768fc9c5db-lrz4s" podUID="7f6103e4-008e-4d58-abbb-473bd3da52f5" containerName="console" containerID="cri-o://954dec5d13a3d0ca18e55caf536660d3eabbea0bdfff3414c8a1233524ac2cf4" gracePeriod=15 Jul 06 12:49:35.278147 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.278115 2559 generic.go:356] "Generic (PLEG): container finished" podID="7f6103e4-008e-4d58-abbb-473bd3da52f5" containerID="954dec5d13a3d0ca18e55caf536660d3eabbea0bdfff3414c8a1233524ac2cf4" exitCode=0 Jul 06 12:49:35.278318 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.278201 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fc9c5db-lrz4s" event={"ID":"7f6103e4-008e-4d58-abbb-473bd3da52f5","Type":"ContainerDied","Data":"954dec5d13a3d0ca18e55caf536660d3eabbea0bdfff3414c8a1233524ac2cf4"} Jul 06 12:49:35.411395 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.411369 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:49:35.455991 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.455965 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg62q\" (UniqueName: \"kubernetes.io/projected/7f6103e4-008e-4d58-abbb-473bd3da52f5-kube-api-access-vg62q\") pod \"7f6103e4-008e-4d58-abbb-473bd3da52f5\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " Jul 06 12:49:35.456138 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456025 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-oauth-config\") pod \"7f6103e4-008e-4d58-abbb-473bd3da52f5\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " Jul 06 12:49:35.456138 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456052 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-oauth-serving-cert\") pod \"7f6103e4-008e-4d58-abbb-473bd3da52f5\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " Jul 06 12:49:35.456138 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456083 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-service-ca\") pod \"7f6103e4-008e-4d58-abbb-473bd3da52f5\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " Jul 06 12:49:35.456345 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456256 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-serving-cert\") pod \"7f6103e4-008e-4d58-abbb-473bd3da52f5\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " Jul 06 12:49:35.456402 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456351 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-config\") pod \"7f6103e4-008e-4d58-abbb-473bd3da52f5\" (UID: \"7f6103e4-008e-4d58-abbb-473bd3da52f5\") " Jul 06 12:49:35.456481 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456450 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7f6103e4-008e-4d58-abbb-473bd3da52f5" (UID: "7f6103e4-008e-4d58-abbb-473bd3da52f5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:35.456481 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456465 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-service-ca" (OuterVolumeSpecName: "service-ca") pod "7f6103e4-008e-4d58-abbb-473bd3da52f5" (UID: "7f6103e4-008e-4d58-abbb-473bd3da52f5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:35.456702 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456682 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-config" (OuterVolumeSpecName: "console-config") pod "7f6103e4-008e-4d58-abbb-473bd3da52f5" (UID: "7f6103e4-008e-4d58-abbb-473bd3da52f5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:35.456857 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456808 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:35.456857 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456829 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-oauth-serving-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:35.456857 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.456844 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6103e4-008e-4d58-abbb-473bd3da52f5-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:35.458402 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.458378 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6103e4-008e-4d58-abbb-473bd3da52f5-kube-api-access-vg62q" (OuterVolumeSpecName: "kube-api-access-vg62q") pod "7f6103e4-008e-4d58-abbb-473bd3da52f5" (UID: "7f6103e4-008e-4d58-abbb-473bd3da52f5"). InnerVolumeSpecName "kube-api-access-vg62q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:49:35.458402 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.458392 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7f6103e4-008e-4d58-abbb-473bd3da52f5" (UID: "7f6103e4-008e-4d58-abbb-473bd3da52f5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:49:35.458530 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.458376 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7f6103e4-008e-4d58-abbb-473bd3da52f5" (UID: "7f6103e4-008e-4d58-abbb-473bd3da52f5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:49:35.557921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.557888 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-serving-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:35.557921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.557919 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg62q\" (UniqueName: \"kubernetes.io/projected/7f6103e4-008e-4d58-abbb-473bd3da52f5-kube-api-access-vg62q\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:35.558134 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:35.557933 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f6103e4-008e-4d58-abbb-473bd3da52f5-console-oauth-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:36.284130 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:36.284103 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fc9c5db-lrz4s" Jul 06 12:49:36.284577 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:36.284098 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fc9c5db-lrz4s" event={"ID":"7f6103e4-008e-4d58-abbb-473bd3da52f5","Type":"ContainerDied","Data":"c4653d8b6d9ed85072f09737b9f97031adc4dcb1cf26cd357da85ac4fe8b341c"} Jul 06 12:49:36.284577 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:36.284237 2559 scope.go:117] "RemoveContainer" containerID="954dec5d13a3d0ca18e55caf536660d3eabbea0bdfff3414c8a1233524ac2cf4" Jul 06 12:49:36.306962 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:36.306929 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768fc9c5db-lrz4s"] Jul 06 12:49:36.309289 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:36.309268 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-768fc9c5db-lrz4s"] Jul 06 12:49:36.414588 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:36.414555 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f6103e4-008e-4d58-abbb-473bd3da52f5" path="/var/lib/kubelet/pods/7f6103e4-008e-4d58-abbb-473bd3da52f5/volumes" Jul 06 12:49:39.258320 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:39.258288 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-stdbh" Jul 06 12:49:46.224530 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.224467 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7dbd46688b-gxz4k" podUID="14045b63-d865-43f6-94bb-ed4eddd6fea7" containerName="console" containerID="cri-o://ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5" gracePeriod=15 Jul 06 12:49:46.489056 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.489000 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:46.547046 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547019 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mlj5\" (UniqueName: \"kubernetes.io/projected/14045b63-d865-43f6-94bb-ed4eddd6fea7-kube-api-access-6mlj5\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547053 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-service-ca\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547074 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-oauth-config\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547112 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-oauth-serving-cert\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547148 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-serving-cert\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547370 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547215 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-config\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547370 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547254 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-trusted-ca-bundle\") pod \"14045b63-d865-43f6-94bb-ed4eddd6fea7\" (UID: \"14045b63-d865-43f6-94bb-ed4eddd6fea7\") " Jul 06 12:49:46.547587 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547558 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-service-ca" (OuterVolumeSpecName: "service-ca") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:46.547681 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547557 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:46.547745 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547678 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-config" (OuterVolumeSpecName: "console-config") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:46.547815 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.547790 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:49:46.549231 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.549205 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:49:46.549666 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.549635 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:49:46.549666 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.549645 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14045b63-d865-43f6-94bb-ed4eddd6fea7-kube-api-access-6mlj5" (OuterVolumeSpecName: "kube-api-access-6mlj5") pod "14045b63-d865-43f6-94bb-ed4eddd6fea7" (UID: "14045b63-d865-43f6-94bb-ed4eddd6fea7"). InnerVolumeSpecName "kube-api-access-6mlj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:49:46.648331 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648303 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-oauth-serving-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:46.648331 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648328 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-serving-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:46.648331 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648338 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:46.648530 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648348 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-trusted-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:46.648530 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648358 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mlj5\" (UniqueName: \"kubernetes.io/projected/14045b63-d865-43f6-94bb-ed4eddd6fea7-kube-api-access-6mlj5\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:46.648530 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648366 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14045b63-d865-43f6-94bb-ed4eddd6fea7-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:46.648530 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:46.648375 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14045b63-d865-43f6-94bb-ed4eddd6fea7-console-oauth-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:49:47.334549 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.334515 2559 generic.go:356] "Generic (PLEG): container finished" podID="14045b63-d865-43f6-94bb-ed4eddd6fea7" containerID="ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5" exitCode=0 Jul 06 12:49:47.334959 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.334558 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dbd46688b-gxz4k" Jul 06 12:49:47.334959 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.334577 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd46688b-gxz4k" event={"ID":"14045b63-d865-43f6-94bb-ed4eddd6fea7","Type":"ContainerDied","Data":"ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5"} Jul 06 12:49:47.334959 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.334617 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dbd46688b-gxz4k" event={"ID":"14045b63-d865-43f6-94bb-ed4eddd6fea7","Type":"ContainerDied","Data":"c77b6d8e83196f151adfd26b94691cf985ce07f2d377e74bb3803c6588cde424"} Jul 06 12:49:47.334959 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.334635 2559 scope.go:117] "RemoveContainer" containerID="ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5" Jul 06 12:49:47.342980 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.342962 2559 scope.go:117] "RemoveContainer" containerID="ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5" Jul 06 12:49:47.343231 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:49:47.343210 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5\": container with ID starting with ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5 not found: ID does not exist" containerID="ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5" Jul 06 12:49:47.343291 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.343240 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5"} err="failed to get container status \"ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5\": rpc error: code = NotFound desc = could not find container \"ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5\": container with ID starting with ee10b47fc07e5165becf57bdb78e990c50828387f56c1fac127dcb9ae422e6d5 not found: ID does not exist" Jul 06 12:49:47.362105 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.362082 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dbd46688b-gxz4k"] Jul 06 12:49:47.367906 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:47.367888 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dbd46688b-gxz4k"] Jul 06 12:49:48.414264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:48.414233 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14045b63-d865-43f6-94bb-ed4eddd6fea7" path="/var/lib/kubelet/pods/14045b63-d865-43f6-94bb-ed4eddd6fea7/volumes" Jul 06 12:49:48.947878 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:48.947850 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:49:48.951638 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:49:48.951616 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-649c995458-8xxxk" Jul 06 12:50:11.137913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:11.137875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:11.157021 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:11.156994 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:11.449255 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:11.449229 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:24.845845 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.845812 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:50:24.846307 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.846098 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="alertmanager" containerID="cri-o://4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08" gracePeriod=120 Jul 06 12:50:24.846307 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.846142 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-metric" containerID="cri-o://9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416" gracePeriod=120 Jul 06 12:50:24.846307 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.846227 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="config-reloader" containerID="cri-o://c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15" gracePeriod=120 Jul 06 12:50:24.846307 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.846240 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy" containerID="cri-o://f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8" gracePeriod=120 Jul 06 12:50:24.846521 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.846215 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-web" containerID="cri-o://ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0" gracePeriod=120 Jul 06 12:50:24.846521 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:24.846235 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="prom-label-proxy" containerID="cri-o://f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2" gracePeriod=120 Jul 06 12:50:25.488772 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488737 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2" exitCode=0 Jul 06 12:50:25.488772 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488765 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8" exitCode=0 Jul 06 12:50:25.488772 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488774 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15" exitCode=0 Jul 06 12:50:25.488772 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488779 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08" exitCode=0 Jul 06 12:50:25.489032 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2"} Jul 06 12:50:25.489032 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488890 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8"} Jul 06 12:50:25.489032 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488918 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15"} Jul 06 12:50:25.489032 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:25.488941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08"} Jul 06 12:50:26.098700 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.098676 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.204847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204764 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.204847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204808 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.204847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204827 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-web\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204870 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-web-config\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204886 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-tls-assets\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204912 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-main-db\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204928 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-volume\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204943 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-metrics-client-ca\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.204977 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-out\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205041 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-cluster-tls-config\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205078 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205124 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205115 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fdkq\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-kube-api-access-6fdkq\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205583 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205152 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy\") pod \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\" (UID: \"1f143d1f-8afa-4db2-96ce-33e4fc6af97a\") " Jul 06 12:50:26.205583 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205231 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:26.205583 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205268 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:50:26.205583 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205474 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.205583 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.205503 2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-alertmanager-main-db\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.206070 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.206041 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:26.207854 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.207818 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.208254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.208209 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:50:26.208352 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.208325 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-kube-api-access-6fdkq" (OuterVolumeSpecName: "kube-api-access-6fdkq") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "kube-api-access-6fdkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:50:26.208456 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.208432 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.208870 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.208848 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.209072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.209046 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-out" (OuterVolumeSpecName: "config-out") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:50:26.209189 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.209144 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.209640 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.209608 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-volume" (OuterVolumeSpecName: "config-volume") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.212037 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.212011 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.218288 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.218265 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-web-config" (OuterVolumeSpecName: "web-config") pod "1f143d1f-8afa-4db2-96ce-33e4fc6af97a" (UID: "1f143d1f-8afa-4db2-96ce-33e4fc6af97a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:26.306137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306085 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306130 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306143 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-web-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306152 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-tls-assets\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306160 2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-volume\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306459 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306191 2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-metrics-client-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306459 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306199 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-config-out\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306459 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306208 2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-cluster-tls-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306459 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306217 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-main-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306459 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306226 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fdkq\" (UniqueName: \"kubernetes.io/projected/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-kube-api-access-6fdkq\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.306459 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.306234 2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f143d1f-8afa-4db2-96ce-33e4fc6af97a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:26.496191 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496087 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416" exitCode=0 Jul 06 12:50:26.496191 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496113 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerID="ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0" exitCode=0 Jul 06 12:50:26.496191 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496159 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416"} Jul 06 12:50:26.496191 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496190 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.496438 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496236 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0"} Jul 06 12:50:26.496438 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496260 2559 scope.go:117] "RemoveContainer" containerID="f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2" Jul 06 12:50:26.496438 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.496264 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1f143d1f-8afa-4db2-96ce-33e4fc6af97a","Type":"ContainerDied","Data":"e31fd5da8787f3a4f1e021fde7935c04552a904395c481848eea7622f9c98dfa"} Jul 06 12:50:26.503863 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.503844 2559 scope.go:117] "RemoveContainer" containerID="9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416" Jul 06 12:50:26.510526 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.510509 2559 scope.go:117] "RemoveContainer" containerID="f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8" Jul 06 12:50:26.517351 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.517336 2559 scope.go:117] "RemoveContainer" containerID="ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0" Jul 06 12:50:26.523887 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.523871 2559 scope.go:117] "RemoveContainer" containerID="c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15" Jul 06 12:50:26.531589 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.531572 2559 scope.go:117] "RemoveContainer" containerID="4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08" Jul 06 12:50:26.532229 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.532209 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:50:26.538226 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.538205 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:50:26.539143 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.539129 2559 scope.go:117] "RemoveContainer" containerID="3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4" Jul 06 12:50:26.545889 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.545876 2559 scope.go:117] "RemoveContainer" containerID="f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2" Jul 06 12:50:26.546143 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.546123 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2\": container with ID starting with f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2 not found: ID does not exist" containerID="f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2" Jul 06 12:50:26.546251 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546148 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2"} err="failed to get container status \"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2\": rpc error: code = NotFound desc = could not find container \"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2\": container with ID starting with f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2 not found: ID does not exist" Jul 06 12:50:26.546251 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546183 2559 scope.go:117] "RemoveContainer" containerID="9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416" Jul 06 12:50:26.546435 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.546418 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416\": container with ID starting with 9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416 not found: ID does not exist" containerID="9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416" Jul 06 12:50:26.546495 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546445 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416"} err="failed to get container status \"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416\": rpc error: code = NotFound desc = could not find container \"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416\": container with ID starting with 9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416 not found: ID does not exist" Jul 06 12:50:26.546495 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546469 2559 scope.go:117] "RemoveContainer" containerID="f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8" Jul 06 12:50:26.546721 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.546705 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8\": container with ID starting with f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8 not found: ID does not exist" containerID="f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8" Jul 06 12:50:26.546768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546726 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8"} err="failed to get container status \"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8\": rpc error: code = NotFound desc = could not find container \"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8\": container with ID starting with f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8 not found: ID does not exist" Jul 06 12:50:26.546768 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546740 2559 scope.go:117] "RemoveContainer" containerID="ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0" Jul 06 12:50:26.546960 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.546944 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0\": container with ID starting with ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0 not found: ID does not exist" containerID="ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0" Jul 06 12:50:26.547007 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546967 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0"} err="failed to get container status \"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0\": rpc error: code = NotFound desc = could not find container \"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0\": container with ID starting with ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0 not found: ID does not exist" Jul 06 12:50:26.547007 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.546991 2559 scope.go:117] "RemoveContainer" containerID="c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15" Jul 06 12:50:26.547238 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.547222 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15\": container with ID starting with c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15 not found: ID does not exist" containerID="c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15" Jul 06 12:50:26.547310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547246 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15"} err="failed to get container status \"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15\": rpc error: code = NotFound desc = could not find container \"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15\": container with ID starting with c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15 not found: ID does not exist" Jul 06 12:50:26.547310 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547265 2559 scope.go:117] "RemoveContainer" containerID="4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08" Jul 06 12:50:26.547478 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.547463 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08\": container with ID starting with 4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08 not found: ID does not exist" containerID="4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08" Jul 06 12:50:26.547517 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547482 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08"} err="failed to get container status \"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08\": rpc error: code = NotFound desc = could not find container \"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08\": container with ID starting with 4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08 not found: ID does not exist" Jul 06 12:50:26.547517 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547494 2559 scope.go:117] "RemoveContainer" containerID="3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4" Jul 06 12:50:26.547720 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:26.547701 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4\": container with ID starting with 3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4 not found: ID does not exist" containerID="3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4" Jul 06 12:50:26.547799 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547727 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4"} err="failed to get container status \"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4\": rpc error: code = NotFound desc = could not find container \"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4\": container with ID starting with 3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4 not found: ID does not exist" Jul 06 12:50:26.547799 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547747 2559 scope.go:117] "RemoveContainer" containerID="f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2" Jul 06 12:50:26.547946 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547930 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2"} err="failed to get container status \"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2\": rpc error: code = NotFound desc = could not find container \"f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2\": container with ID starting with f85730cde7d5463d0e5a8aeb713473594cb94f0439b753c4ad56f34d1f62dfa2 not found: ID does not exist" Jul 06 12:50:26.548009 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.547948 2559 scope.go:117] "RemoveContainer" containerID="9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416" Jul 06 12:50:26.548148 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548133 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416"} err="failed to get container status \"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416\": rpc error: code = NotFound desc = could not find container \"9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416\": container with ID starting with 9dc7b05d106d5e27266c4541af7a1ba6a3ed67a44830737a0819c5b114efd416 not found: ID does not exist" Jul 06 12:50:26.548232 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548149 2559 scope.go:117] "RemoveContainer" containerID="f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8" Jul 06 12:50:26.548394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548377 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8"} err="failed to get container status \"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8\": rpc error: code = NotFound desc = could not find container \"f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8\": container with ID starting with f8beb9b0fed5f33c957218696fee909f75d7a4d2854f7ee49990366b69c437b8 not found: ID does not exist" Jul 06 12:50:26.548440 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548395 2559 scope.go:117] "RemoveContainer" containerID="ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0" Jul 06 12:50:26.548590 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548571 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0"} err="failed to get container status \"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0\": rpc error: code = NotFound desc = could not find container \"ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0\": container with ID starting with ba6f34b3d7799d3bf5c689b5f5fb3babb15417797581bd8241f2f7891e1b79f0 not found: ID does not exist" Jul 06 12:50:26.548655 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548593 2559 scope.go:117] "RemoveContainer" containerID="c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15" Jul 06 12:50:26.548787 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548770 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15"} err="failed to get container status \"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15\": rpc error: code = NotFound desc = could not find container \"c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15\": container with ID starting with c7c92eee81a2391de0b3d3cbf5d8656a87b4e2bd66af280c0eda7ddef6c41b15 not found: ID does not exist" Jul 06 12:50:26.548854 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548788 2559 scope.go:117] "RemoveContainer" containerID="4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08" Jul 06 12:50:26.548987 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548969 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08"} err="failed to get container status \"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08\": rpc error: code = NotFound desc = could not find container \"4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08\": container with ID starting with 4356c8224534a0ec40dc866c87f4576dca0153ec48e49241af5d6c447e400d08 not found: ID does not exist" Jul 06 12:50:26.549044 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.548987 2559 scope.go:117] "RemoveContainer" containerID="3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4" Jul 06 12:50:26.549191 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.549175 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4"} err="failed to get container status \"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4\": rpc error: code = NotFound desc = could not find container \"3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4\": container with ID starting with 3db099fc405a56508b581a9c7f675570b0cf4d29887a863c1291a9f194e75ea4 not found: ID does not exist" Jul 06 12:50:26.573646 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573622 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:50:26.573912 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573899 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14045b63-d865-43f6-94bb-ed4eddd6fea7" containerName="console" Jul 06 12:50:26.573949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573914 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14045b63-d865-43f6-94bb-ed4eddd6fea7" containerName="console" Jul 06 12:50:26.573949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573924 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-web" Jul 06 12:50:26.573949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573929 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-web" Jul 06 12:50:26.573949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573937 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6103e4-008e-4d58-abbb-473bd3da52f5" containerName="console" Jul 06 12:50:26.573949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573942 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6103e4-008e-4d58-abbb-473bd3da52f5" containerName="console" Jul 06 12:50:26.573949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573949 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="init-config-reloader" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573954 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="init-config-reloader" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573961 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573966 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573976 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-metric" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573981 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-metric" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573990 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="alertmanager" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.573995 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="alertmanager" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574003 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="config-reloader" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574008 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="config-reloader" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574013 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="prom-label-proxy" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574018 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="prom-label-proxy" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574075 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="config-reloader" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574082 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574089 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6103e4-008e-4d58-abbb-473bd3da52f5" containerName="console" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574096 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="14045b63-d865-43f6-94bb-ed4eddd6fea7" containerName="console" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574102 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-web" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574106 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="alertmanager" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574111 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="prom-label-proxy" Jul 06 12:50:26.574116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.574117 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" containerName="kube-rbac-proxy-metric" Jul 06 12:50:26.577451 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.577437 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.580254 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.580225 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 12:50:26.580357 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.580336 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 12:50:26.580410 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.580336 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 12:50:26.580818 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.580799 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 12:50:26.580998 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.580973 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 12:50:26.581154 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.581138 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 12:50:26.581264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.581224 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-45777\"" Jul 06 12:50:26.581380 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.581360 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 12:50:26.581578 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.581559 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 12:50:26.587919 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.587901 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 12:50:26.600474 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.600454 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:50:26.708480 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708446 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-config-out\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708480 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708683 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-web-config\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708683 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708587 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708683 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708619 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708683 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708683 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708683 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708679 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708701 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708818 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708854 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708887 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpd2t\" (UniqueName: \"kubernetes.io/projected/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-kube-api-access-gpd2t\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.708949 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.708928 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.809780 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-config-out\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.809780 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.809780 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809776 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-web-config\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809827 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809854 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809927 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.809975 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.810000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.810029 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpd2t\" (UniqueName: \"kubernetes.io/projected/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-kube-api-access-gpd2t\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.810068 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.810065 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.811209 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.811117 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.811679 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.811644 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.812269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.812247 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813110 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813084 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-web-config\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813226 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813211 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813286 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813257 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813436 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813420 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813588 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813749 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813729 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.813799 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813758 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.814004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.813985 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-config-out\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.814549 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.814531 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.820025 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.820005 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpd2t\" (UniqueName: \"kubernetes.io/projected/9d2d4aa7-1246-425d-94a1-59352e5ac1b4-kube-api-access-gpd2t\") pod \"alertmanager-main-0\" (UID: \"9d2d4aa7-1246-425d-94a1-59352e5ac1b4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:26.887712 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:26.887678 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 12:50:27.029724 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:27.029699 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 12:50:27.033154 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:50:27.033032 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2d4aa7_1246_425d_94a1_59352e5ac1b4.slice/crio-063c1b75a1e861038ac6bfd6df05bb33161a45a44fd9dc17092ebe5e290eb996 WatchSource:0}: Error finding container 063c1b75a1e861038ac6bfd6df05bb33161a45a44fd9dc17092ebe5e290eb996: Status 404 returned error can't find the container with id 063c1b75a1e861038ac6bfd6df05bb33161a45a44fd9dc17092ebe5e290eb996 Jul 06 12:50:27.502455 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:27.502423 2559 generic.go:356] "Generic (PLEG): container finished" podID="9d2d4aa7-1246-425d-94a1-59352e5ac1b4" containerID="fde9d8970c4f07e066d0742a0e2285d3f51c986612dff84969f3dbdee9cf3791" exitCode=0 Jul 06 12:50:27.502868 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:27.502483 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerDied","Data":"fde9d8970c4f07e066d0742a0e2285d3f51c986612dff84969f3dbdee9cf3791"} Jul 06 12:50:27.502868 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:27.502563 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"063c1b75a1e861038ac6bfd6df05bb33161a45a44fd9dc17092ebe5e290eb996"} Jul 06 12:50:28.414823 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.414780 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f143d1f-8afa-4db2-96ce-33e4fc6af97a" path="/var/lib/kubelet/pods/1f143d1f-8afa-4db2-96ce-33e4fc6af97a/volumes" Jul 06 12:50:28.512970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.512939 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"9ca9470902b9314208277b11009649694d32f83166c6aa7a57c695258862323f"} Jul 06 12:50:28.513369 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.513002 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"20e2dd27df26549d4c94bde1c97cc0c1de74bf0d0a3546e6ff818f8e4064f450"} Jul 06 12:50:28.513369 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.513030 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"3a7bfedfda0f7ad55aff8ed2e2e4ebe79afa8383543197b6100f0f4f77ab77b9"} Jul 06 12:50:28.513369 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.513052 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"3d97175123c869a6270ffe2c63ed68fd2f30fe097f258f655267629a9e57e602"} Jul 06 12:50:28.513369 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.513074 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"2776d17fa87f7642ec21b9a29cd800b77bc1ecc244df793c7d0573ab51351506"} Jul 06 12:50:28.513369 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.513097 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d2d4aa7-1246-425d-94a1-59352e5ac1b4","Type":"ContainerStarted","Data":"4d75ed50a2ec449a91d3dc6679adeefd4d2cb203a10a9f819eec181f814f1d6a"} Jul 06 12:50:28.548704 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.548640 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.5382272390000002 podStartE2EDuration="2.548615167s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.387928ms" isStatefulPod=false podCreationTimestamp="2026-07-06 12:50:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:50:28.548156679 +0000 UTC m=+228.629597734" watchObservedRunningTime="2026-07-06 12:50:28.548615167 +0000 UTC m=+228.630056196" Jul 06 12:50:28.830815 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.830783 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-cf5477557-d76bd"] Jul 06 12:50:28.833541 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.833522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.837151 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.837125 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 12:50:28.837295 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.837284 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 12:50:28.837382 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.837368 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 12:50:28.837501 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.837479 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 12:50:28.838288 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.838239 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wxrks\"" Jul 06 12:50:28.838543 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.838529 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 12:50:28.844820 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.844797 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 12:50:28.861557 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.861524 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cf5477557-d76bd"] Jul 06 12:50:28.929455 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-serving-certs-ca-bundle\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929598 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929464 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-metrics-client-ca\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929598 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929525 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929598 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929565 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-secret-telemeter-client\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929709 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929611 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-federate-client-tls\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929709 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929709 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929685 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-telemeter-client-tls\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:28.929807 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:28.929717 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rzw4\" (UniqueName: \"kubernetes.io/projected/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-kube-api-access-8rzw4\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030108 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-serving-certs-ca-bundle\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030303 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030120 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-metrics-client-ca\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030303 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030383 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-secret-telemeter-client\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030447 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030430 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-federate-client-tls\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030502 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030463 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030558 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-telemeter-client-tls\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030614 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rzw4\" (UniqueName: \"kubernetes.io/projected/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-kube-api-access-8rzw4\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.030933 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030904 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-metrics-client-ca\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.031044 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.030936 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-serving-certs-ca-bundle\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.031258 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.031236 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.032987 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.032964 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-telemeter-client-tls\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.033071 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.033023 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-federate-client-tls\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.033388 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.033367 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.033434 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.033392 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-secret-telemeter-client\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.039539 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.039522 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rzw4\" (UniqueName: \"kubernetes.io/projected/10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa-kube-api-access-8rzw4\") pod \"telemeter-client-cf5477557-d76bd\" (UID: \"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa\") " pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.143491 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.143405 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" Jul 06 12:50:29.149754 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.149727 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:50:29.150468 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.150435 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="prometheus" containerID="cri-o://3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" gracePeriod=600 Jul 06 12:50:29.151264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.151032 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" gracePeriod=600 Jul 06 12:50:29.151264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.151199 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy" containerID="cri-o://35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" gracePeriod=600 Jul 06 12:50:29.151407 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.151309 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-web" containerID="cri-o://e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" gracePeriod=600 Jul 06 12:50:29.151462 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.151416 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="thanos-sidecar" containerID="cri-o://9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" gracePeriod=600 Jul 06 12:50:29.151577 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.151522 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="config-reloader" containerID="cri-o://8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" gracePeriod=600 Jul 06 12:50:29.319296 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.319267 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cf5477557-d76bd"] Jul 06 12:50:29.321765 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:50:29.321732 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10521b1b_6ef5_4d1d_9e57_b0285cfbf1fa.slice/crio-602efc7fc161745be256902d64c9b9a3d1a8fba256154a531f67991279f38813 WatchSource:0}: Error finding container 602efc7fc161745be256902d64c9b9a3d1a8fba256154a531f67991279f38813: Status 404 returned error can't find the container with id 602efc7fc161745be256902d64c9b9a3d1a8fba256154a531f67991279f38813 Jul 06 12:50:29.423253 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.423229 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.518248 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.518135 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" event={"ID":"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa","Type":"ContainerStarted","Data":"602efc7fc161745be256902d64c9b9a3d1a8fba256154a531f67991279f38813"} Jul 06 12:50:29.521228 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521187 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" exitCode=0 Jul 06 12:50:29.521228 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521227 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" exitCode=0 Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521234 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" exitCode=0 Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521240 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" exitCode=0 Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521246 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" exitCode=0 Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521251 2559 generic.go:356] "Generic (PLEG): container finished" podID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" exitCode=0 Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521263 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521232 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521351 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521378 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} Jul 06 12:50:29.521408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521402 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} Jul 06 12:50:29.521720 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521413 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.521720 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} Jul 06 12:50:29.521720 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521451 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} Jul 06 12:50:29.521720 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.521474 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8fafa7a5-b356-417a-8d1e-251cf404febe","Type":"ContainerDied","Data":"e1612ef31a51f82039fdd00dc2fb7a394357cea7a099e61d4a8f1fe8560dae74"} Jul 06 12:50:29.529197 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.529154 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.535418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535385 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-config\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535534 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535446 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-1\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535534 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535486 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535534 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535511 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-config-out\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535539 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-grpc-tls\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535561 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-web-config\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535587 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-trusted-ca-bundle\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535614 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-0\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535693 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535648 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-2\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535715 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-db\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535740 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-metrics-client-ca\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535777 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-thanos-prometheus-http-client-file\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535803 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535816 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535862 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-metrics-client-certs\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535897 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9kh8\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-kube-api-access-m9kh8\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.535941 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535931 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-tls-assets\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.536394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535958 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-kubelet-serving-ca-bundle\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.536394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.535983 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-kube-rbac-proxy\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.536394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536010 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-tls\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.536394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536045 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-serving-certs-ca-bundle\") pod \"8fafa7a5-b356-417a-8d1e-251cf404febe\" (UID: \"8fafa7a5-b356-417a-8d1e-251cf404febe\") " Jul 06 12:50:29.536394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536087 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.536394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536194 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.536674 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536443 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.536674 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536486 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.536674 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536508 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.536674 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536523 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-metrics-client-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.536871 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.536803 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.537612 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.537582 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:50:29.537726 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.537632 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.538361 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.538066 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 12:50:29.538361 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.538108 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.539452 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.539234 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.539582 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.539508 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-config" (OuterVolumeSpecName: "config") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.539582 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.539559 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.539702 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.539613 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:50:29.539702 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.539632 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-config-out" (OuterVolumeSpecName: "config-out") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 12:50:29.539871 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.539774 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-kube-api-access-m9kh8" (OuterVolumeSpecName: "kube-api-access-m9kh8") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "kube-api-access-m9kh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:50:29.540772 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.540746 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.541301 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.541274 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.541301 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.541289 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.541437 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.541417 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.541907 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.541888 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.546511 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.546494 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.551738 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.551717 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-web-config" (OuterVolumeSpecName: "web-config") pod "8fafa7a5-b356-417a-8d1e-251cf404febe" (UID: "8fafa7a5-b356-417a-8d1e-251cf404febe"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:50:29.553561 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.553546 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.560037 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.560015 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.567374 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.567356 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.573847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.573830 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.574101 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.574082 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.574142 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574111 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} err="failed to get container status \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" Jul 06 12:50:29.574142 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574131 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.574363 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.574349 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.574407 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574366 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} err="failed to get container status \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" Jul 06 12:50:29.574407 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574379 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.574603 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.574585 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.574641 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574610 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} err="failed to get container status \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" Jul 06 12:50:29.574641 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574626 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.574821 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.574805 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.574886 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574830 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} err="failed to get container status \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" Jul 06 12:50:29.574886 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.574851 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.575082 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.575067 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.575116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575087 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} err="failed to get container status \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" Jul 06 12:50:29.575116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575099 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.575336 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.575308 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.575384 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575333 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} err="failed to get container status \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" Jul 06 12:50:29.575384 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575350 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.575550 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:50:29.575535 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.575586 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575554 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} err="failed to get container status \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" Jul 06 12:50:29.575586 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575568 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.575796 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575771 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} err="failed to get container status \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" Jul 06 12:50:29.575844 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575797 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.575993 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575979 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} err="failed to get container status \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" Jul 06 12:50:29.576032 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.575993 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.576211 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576191 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} err="failed to get container status \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" Jul 06 12:50:29.576264 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576213 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.576422 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576407 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} err="failed to get container status \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" Jul 06 12:50:29.576422 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576421 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.576619 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576603 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} err="failed to get container status \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" Jul 06 12:50:29.576664 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576619 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.576816 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576797 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} err="failed to get container status \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" Jul 06 12:50:29.576884 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.576817 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.577019 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577004 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} err="failed to get container status \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" Jul 06 12:50:29.577055 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577019 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.577245 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577227 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} err="failed to get container status \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" Jul 06 12:50:29.577294 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577246 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.577455 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577439 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} err="failed to get container status \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" Jul 06 12:50:29.577506 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577455 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.577680 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577663 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} err="failed to get container status \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" Jul 06 12:50:29.577720 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577681 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.577887 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577870 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} err="failed to get container status \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" Jul 06 12:50:29.577952 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.577890 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.578109 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578095 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} err="failed to get container status \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" Jul 06 12:50:29.578153 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578110 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.578309 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578291 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} err="failed to get container status \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" Jul 06 12:50:29.578349 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578309 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.578493 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578476 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} err="failed to get container status \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" Jul 06 12:50:29.578537 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578493 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.578668 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578651 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} err="failed to get container status \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" Jul 06 12:50:29.578705 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578668 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.578831 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578815 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} err="failed to get container status \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" Jul 06 12:50:29.578874 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578831 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.578998 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578984 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} err="failed to get container status \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" Jul 06 12:50:29.579039 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.578999 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.579204 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579189 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} err="failed to get container status \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" Jul 06 12:50:29.579204 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579204 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.579412 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579396 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} err="failed to get container status \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" Jul 06 12:50:29.579466 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579412 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.579625 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579610 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} err="failed to get container status \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" Jul 06 12:50:29.579625 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579625 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.579812 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579795 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} err="failed to get container status \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" Jul 06 12:50:29.579853 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.579811 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.580022 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580007 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} err="failed to get container status \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" Jul 06 12:50:29.580080 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580022 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.580249 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580230 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} err="failed to get container status \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" Jul 06 12:50:29.580249 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580248 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.580467 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580445 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} err="failed to get container status \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" Jul 06 12:50:29.580512 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580468 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.580695 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580679 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} err="failed to get container status \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" Jul 06 12:50:29.580742 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580696 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.580877 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580862 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} err="failed to get container status \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" Jul 06 12:50:29.580935 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.580878 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.581073 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581059 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} err="failed to get container status \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" Jul 06 12:50:29.581113 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581073 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.581274 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581256 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} err="failed to get container status \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" Jul 06 12:50:29.581358 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581274 2559 scope.go:117] "RemoveContainer" containerID="9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7" Jul 06 12:50:29.581449 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581434 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7"} err="failed to get container status \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": rpc error: code = NotFound desc = could not find container \"9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7\": container with ID starting with 9f16c6a347e91c3f93f0f06427c30250cac4094f89459d369876c96cc23306a7 not found: ID does not exist" Jul 06 12:50:29.581485 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581450 2559 scope.go:117] "RemoveContainer" containerID="35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38" Jul 06 12:50:29.581608 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581593 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38"} err="failed to get container status \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": rpc error: code = NotFound desc = could not find container \"35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38\": container with ID starting with 35f631e2a36205b9248b55f9bc009a79429559b36d51e190b7ba8e3f7b3e0d38 not found: ID does not exist" Jul 06 12:50:29.581647 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581609 2559 scope.go:117] "RemoveContainer" containerID="e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270" Jul 06 12:50:29.581809 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581794 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270"} err="failed to get container status \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": rpc error: code = NotFound desc = could not find container \"e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270\": container with ID starting with e72846ab16cb0ecd44858415bd060945e5d12455bbe4b0221f2c18b3f099b270 not found: ID does not exist" Jul 06 12:50:29.581852 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.581809 2559 scope.go:117] "RemoveContainer" containerID="9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559" Jul 06 12:50:29.582019 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582003 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559"} err="failed to get container status \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": rpc error: code = NotFound desc = could not find container \"9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559\": container with ID starting with 9923fc3188ed02e65a408534801e08f124b216a0d02275c2fa4f34830956d559 not found: ID does not exist" Jul 06 12:50:29.582019 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582018 2559 scope.go:117] "RemoveContainer" containerID="8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face" Jul 06 12:50:29.582177 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582151 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face"} err="failed to get container status \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": rpc error: code = NotFound desc = could not find container \"8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face\": container with ID starting with 8928ec0d83182c9fa8ae80da222ef14fdc956a05f6d280e9dc84017aa410face not found: ID does not exist" Jul 06 12:50:29.582230 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582186 2559 scope.go:117] "RemoveContainer" containerID="3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01" Jul 06 12:50:29.582386 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582365 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01"} err="failed to get container status \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": rpc error: code = NotFound desc = could not find container \"3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01\": container with ID starting with 3049a892e84c5f1cd349ae846d0a2d43155c45c0b7e1ea648635a1396301fb01 not found: ID does not exist" Jul 06 12:50:29.582430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582386 2559 scope.go:117] "RemoveContainer" containerID="dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c" Jul 06 12:50:29.582579 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.582562 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c"} err="failed to get container status \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": rpc error: code = NotFound desc = could not find container \"dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c\": container with ID starting with dfe03f34a9e7495fd030fd3d264376f679b99de4ef5ac3e37bd8eadd9fa8e97c not found: ID does not exist" Jul 06 12:50:29.637592 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637493 2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-metrics-client-certs\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637592 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637526 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9kh8\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-kube-api-access-m9kh8\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637592 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637544 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8fafa7a5-b356-417a-8d1e-251cf404febe-tls-assets\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637592 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637560 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637592 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637574 2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-kube-rbac-proxy\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637592 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637588 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637611 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637631 2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637646 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637662 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-config-out\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637677 2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-grpc-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637690 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-web-config\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637704 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637721 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637735 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8fafa7a5-b356-417a-8d1e-251cf404febe-prometheus-k8s-db\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637751 2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.637915 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.637766 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8fafa7a5-b356-417a-8d1e-251cf404febe-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:50:29.846659 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.846622 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:50:29.853038 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.853009 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:50:29.887694 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.887657 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:50:29.888035 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888017 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-web" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888039 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-web" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888050 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-thanos" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888058 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-thanos" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888077 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="init-config-reloader" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888086 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="init-config-reloader" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888100 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="thanos-sidecar" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888110 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="thanos-sidecar" Jul 06 12:50:29.888118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888118 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888127 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888146 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="config-reloader" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888156 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="config-reloader" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888195 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="prometheus" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888204 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="prometheus" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888289 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="config-reloader" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888303 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888314 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="thanos-sidecar" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888322 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-thanos" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888331 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="kube-rbac-proxy-web" Jul 06 12:50:29.888504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.888341 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" containerName="prometheus" Jul 06 12:50:29.892183 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.892145 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.895501 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.895481 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 12:50:29.895893 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.895876 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-dcwpq\"" Jul 06 12:50:29.895986 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.895928 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 12:50:29.896118 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896098 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 12:50:29.896330 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896314 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 12:50:29.896418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896367 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 12:50:29.896603 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896586 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 12:50:29.896680 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896615 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 12:50:29.896800 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896785 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 12:50:29.896847 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896829 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 12:50:29.897000 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.896988 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 12:50:29.897239 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.897220 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 12:50:29.897275 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.897247 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 12:50:29.897660 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.897644 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7icfv464d1s36\"" Jul 06 12:50:29.917306 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.917271 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 12:50:29.939629 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.939598 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 12:50:29.940686 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940665 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.940804 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-web-config\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.940804 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940727 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.940804 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940788 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-config\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.940928 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-config-out\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.940928 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.940928 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940871 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941048 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940949 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941048 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.940988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941048 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941012 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941048 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941030 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941048 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941045 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941065 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941111 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941127 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941147 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941208 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941238 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941470 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.941470 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.941292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q22j\" (UniqueName: \"kubernetes.io/projected/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-kube-api-access-7q22j\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:29.950701 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:29.950672 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:50:30.042572 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042547 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.042572 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042582 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-config\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.042759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042617 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-config-out\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.042759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042641 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.042759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.042759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042717 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.042759 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042744 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042793 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042895 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042955 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.042998 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043021 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043067 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q22j\" (UniqueName: \"kubernetes.io/projected/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-kube-api-access-7q22j\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043123 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043153 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-web-config\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043430 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.043718 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.043506 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.044435 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.044108 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.045702 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.045435 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.046598 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.046121 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-config-out\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.046598 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.046249 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.046598 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.046278 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-web-config\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.047318 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.047262 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.048320 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.048058 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.048320 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.048274 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.048320 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.048285 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.048504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.048397 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.048565 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.048543 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.048997 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.048939 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.049113 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.049094 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-config\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.049113 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.049104 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.049219 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.049113 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.049571 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.049550 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.050366 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.050347 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.053088 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.053063 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q22j\" (UniqueName: \"kubernetes.io/projected/1c6ecfb7-3fda-4067-9f5b-97cc2222b28f-kube-api-access-7q22j\") pod \"prometheus-k8s-0\" (UID: \"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.201825 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.201788 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:50:30.352640 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.346962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 12:50:30.352913 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:50:30.352883 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6ecfb7_3fda_4067_9f5b_97cc2222b28f.slice/crio-95a0747ac75dc7e6b4656fbc058b1a29fc052406419ed9b71d140c5ae77f1ce8 WatchSource:0}: Error finding container 95a0747ac75dc7e6b4656fbc058b1a29fc052406419ed9b71d140c5ae77f1ce8: Status 404 returned error can't find the container with id 95a0747ac75dc7e6b4656fbc058b1a29fc052406419ed9b71d140c5ae77f1ce8 Jul 06 12:50:30.415792 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.415756 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fafa7a5-b356-417a-8d1e-251cf404febe" path="/var/lib/kubelet/pods/8fafa7a5-b356-417a-8d1e-251cf404febe/volumes" Jul 06 12:50:30.531023 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.530927 2559 generic.go:356] "Generic (PLEG): container finished" podID="1c6ecfb7-3fda-4067-9f5b-97cc2222b28f" containerID="dca42b170bb1c5abf4f309095a542c55d23d572265c731c05017ec1a4f06742e" exitCode=0 Jul 06 12:50:30.531414 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.531019 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerDied","Data":"dca42b170bb1c5abf4f309095a542c55d23d572265c731c05017ec1a4f06742e"} Jul 06 12:50:30.531414 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:30.531102 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"95a0747ac75dc7e6b4656fbc058b1a29fc052406419ed9b71d140c5ae77f1ce8"} Jul 06 12:50:31.538875 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.538836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"c1fa0100224a5f42458bfc70b570306832e1f81b6cfcf89f5d7c6d2caf04128b"} Jul 06 12:50:31.539269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.538918 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"72f380472c22140a50d243c7db66004c55cae2aba77755feeab676e679042bc0"} Jul 06 12:50:31.539269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.538961 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"5d405af66aae89f74afe7ee1bdc2f82c6120c22613358290d6c2b14e47973533"} Jul 06 12:50:31.539269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.539004 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"d5a1c60c1e488e58c430b4fc8861a865c0cc00360d8596b24dd27af223dec35a"} Jul 06 12:50:31.539269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.539040 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"8c913971bf59d8b46417a2103c08a23245110f979eb78ed453018c82ab318b35"} Jul 06 12:50:31.539269 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.539073 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1c6ecfb7-3fda-4067-9f5b-97cc2222b28f","Type":"ContainerStarted","Data":"bd5df384ff6a1cc2220db0e1f260e1ef7b51619e7ed28221e930ae8904356da0"} Jul 06 12:50:31.578803 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:31.577925 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.557398261 podStartE2EDuration="2.577901023s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.502762ms" isStatefulPod=false podCreationTimestamp="2026-07-06 12:50:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:50:31.575570761 +0000 UTC m=+231.657011789" watchObservedRunningTime="2026-07-06 12:50:31.577901023 +0000 UTC m=+231.659342051" Jul 06 12:50:32.544943 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:32.544903 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" event={"ID":"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa","Type":"ContainerStarted","Data":"bcd44407674d4b431c8ce92831bcdb6dcc7708ec0f08933920ba38a0762548e2"} Jul 06 12:50:32.545429 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:32.544977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" event={"ID":"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa","Type":"ContainerStarted","Data":"245c6a36287292e416bee191790af0a6c4f2980513466f9f048972b5ac63e476"} Jul 06 12:50:32.545429 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:32.545004 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" event={"ID":"10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa","Type":"ContainerStarted","Data":"27df1d391fd3228a823ebf851b7cb7b100a21a9f983de3a88a787fe87fc45f01"} Jul 06 12:50:32.573831 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:32.573781 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-cf5477557-d76bd" podStartSLOduration=1.895116932 podStartE2EDuration="4.573761929s" totalImagesPullingTime="2.678644997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:50:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:50:32.572558416 +0000 UTC m=+232.653999474" watchObservedRunningTime="2026-07-06 12:50:32.573761929 +0000 UTC m=+232.655202958" Jul 06 12:50:35.202144 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:50:35.202085 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:51:30.202996 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:30.202949 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:51:30.218522 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:30.218492 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:51:30.783679 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:30.783651 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 12:51:40.340386 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:40.340354 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:51:40.344873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:40.344853 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:51:40.345072 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:40.345054 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:51:40.349737 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:40.349705 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:51:40.351843 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:51:40.351821 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 12:54:56.059016 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.058980 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-94vkj"] Jul 06 12:54:56.062116 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.062100 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.064964 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.064939 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 12:54:56.065802 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.065780 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-47wpc\"" Jul 06 12:54:56.065802 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.065790 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 12:54:56.073633 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.073612 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-94vkj"] Jul 06 12:54:56.149557 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.149520 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9j4r\" (UniqueName: \"kubernetes.io/projected/26b0dbad-a73b-4c60-b3b4-239cd8125e20-kube-api-access-l9j4r\") pod \"cert-manager-cainjector-559cdd8c87-94vkj\" (UID: \"26b0dbad-a73b-4c60-b3b4-239cd8125e20\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.149745 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.149590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26b0dbad-a73b-4c60-b3b4-239cd8125e20-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-94vkj\" (UID: \"26b0dbad-a73b-4c60-b3b4-239cd8125e20\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.250675 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.250630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9j4r\" (UniqueName: \"kubernetes.io/projected/26b0dbad-a73b-4c60-b3b4-239cd8125e20-kube-api-access-l9j4r\") pod \"cert-manager-cainjector-559cdd8c87-94vkj\" (UID: \"26b0dbad-a73b-4c60-b3b4-239cd8125e20\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.250850 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.250707 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26b0dbad-a73b-4c60-b3b4-239cd8125e20-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-94vkj\" (UID: \"26b0dbad-a73b-4c60-b3b4-239cd8125e20\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.259856 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.259830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26b0dbad-a73b-4c60-b3b4-239cd8125e20-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-94vkj\" (UID: \"26b0dbad-a73b-4c60-b3b4-239cd8125e20\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.259980 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.259957 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9j4r\" (UniqueName: \"kubernetes.io/projected/26b0dbad-a73b-4c60-b3b4-239cd8125e20-kube-api-access-l9j4r\") pod \"cert-manager-cainjector-559cdd8c87-94vkj\" (UID: \"26b0dbad-a73b-4c60-b3b4-239cd8125e20\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.381384 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.381278 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" Jul 06 12:54:56.520146 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.520113 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-94vkj"] Jul 06 12:54:56.523470 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:54:56.523442 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26b0dbad_a73b_4c60_b3b4_239cd8125e20.slice/crio-bbf3b69be914f8c5ad38aadedd4663789917bfcc23266fcb306ac59dc056a54e WatchSource:0}: Error finding container bbf3b69be914f8c5ad38aadedd4663789917bfcc23266fcb306ac59dc056a54e: Status 404 returned error can't find the container with id bbf3b69be914f8c5ad38aadedd4663789917bfcc23266fcb306ac59dc056a54e Jul 06 12:54:56.525561 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.525535 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 12:54:56.578060 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:54:56.578026 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" event={"ID":"26b0dbad-a73b-4c60-b3b4-239cd8125e20","Type":"ContainerStarted","Data":"bbf3b69be914f8c5ad38aadedd4663789917bfcc23266fcb306ac59dc056a54e"} Jul 06 12:55:00.596996 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:00.596962 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" event={"ID":"26b0dbad-a73b-4c60-b3b4-239cd8125e20","Type":"ContainerStarted","Data":"cad334c931425108d1501d1860cec9671becc40f624aea9aa68d58384d633276"} Jul 06 12:55:00.616067 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:00.616009 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-94vkj" podStartSLOduration=1.3999622760000001 podStartE2EDuration="4.615992436s" totalImagesPullingTime="3.21603016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:54:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:55:00.615067963 +0000 UTC m=+500.696508992" watchObservedRunningTime="2026-07-06 12:55:00.615992436 +0000 UTC m=+500.697433463" Jul 06 12:55:24.502969 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.502935 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8"] Jul 06 12:55:24.506146 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.506129 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.513159 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.513133 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 12:55:24.513316 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.513282 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 12:55:24.513501 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.513479 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 12:55:24.513602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.513509 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t7n7s\"" Jul 06 12:55:24.513602 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.513522 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 12:55:24.530139 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.530113 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8"] Jul 06 12:55:24.596642 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.596608 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrd4r\" (UniqueName: \"kubernetes.io/projected/86603566-35bd-4275-929e-795ca6acdf60-kube-api-access-qrd4r\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.596769 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.596651 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86603566-35bd-4275-929e-795ca6acdf60-webhook-cert\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.596769 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.596679 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86603566-35bd-4275-929e-795ca6acdf60-manifests\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.596860 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.596785 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86603566-35bd-4275-929e-795ca6acdf60-tmp\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.596860 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.596827 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86603566-35bd-4275-929e-795ca6acdf60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.697229 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.697200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86603566-35bd-4275-929e-795ca6acdf60-tmp\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.697354 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.697241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86603566-35bd-4275-929e-795ca6acdf60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.697354 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.697301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrd4r\" (UniqueName: \"kubernetes.io/projected/86603566-35bd-4275-929e-795ca6acdf60-kube-api-access-qrd4r\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.697354 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.697334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86603566-35bd-4275-929e-795ca6acdf60-webhook-cert\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.697500 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.697361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86603566-35bd-4275-929e-795ca6acdf60-manifests\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.701186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.698090 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86603566-35bd-4275-929e-795ca6acdf60-manifests\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.701186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.700351 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86603566-35bd-4275-929e-795ca6acdf60-tmp\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.701186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.700684 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86603566-35bd-4275-929e-795ca6acdf60-webhook-cert\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.701186 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.700684 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86603566-35bd-4275-929e-795ca6acdf60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.713596 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.713574 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrd4r\" (UniqueName: \"kubernetes.io/projected/86603566-35bd-4275-929e-795ca6acdf60-kube-api-access-qrd4r\") pod \"opendatahub-operator-controller-manager-58899bc45d-ts8m8\" (UID: \"86603566-35bd-4275-929e-795ca6acdf60\") " pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.816913 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.816829 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:24.960445 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:24.960411 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8"] Jul 06 12:55:24.964521 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:55:24.964492 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86603566_35bd_4275_929e_795ca6acdf60.slice/crio-b0ac7d84453ad19c21ee4ff0a6775352f399db760c8ef12408372ecc18d1cd60 WatchSource:0}: Error finding container b0ac7d84453ad19c21ee4ff0a6775352f399db760c8ef12408372ecc18d1cd60: Status 404 returned error can't find the container with id b0ac7d84453ad19c21ee4ff0a6775352f399db760c8ef12408372ecc18d1cd60 Jul 06 12:55:25.698635 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:25.698597 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" event={"ID":"86603566-35bd-4275-929e-795ca6acdf60","Type":"ContainerStarted","Data":"b0ac7d84453ad19c21ee4ff0a6775352f399db760c8ef12408372ecc18d1cd60"} Jul 06 12:55:28.713522 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:28.713489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" event={"ID":"86603566-35bd-4275-929e-795ca6acdf60","Type":"ContainerStarted","Data":"9cc6d998cfc0c24a93048959ebc4566c2519749d3bbda674a517f8c56510669d"} Jul 06 12:55:29.719221 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:29.719187 2559 generic.go:356] "Generic (PLEG): container finished" podID="86603566-35bd-4275-929e-795ca6acdf60" containerID="9cc6d998cfc0c24a93048959ebc4566c2519749d3bbda674a517f8c56510669d" exitCode=0 Jul 06 12:55:29.719637 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:29.719250 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" event={"ID":"86603566-35bd-4275-929e-795ca6acdf60","Type":"ContainerDied","Data":"9cc6d998cfc0c24a93048959ebc4566c2519749d3bbda674a517f8c56510669d"} Jul 06 12:55:30.724608 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:30.724567 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" event={"ID":"86603566-35bd-4275-929e-795ca6acdf60","Type":"ContainerStarted","Data":"c1111c03b72fcbde187c97779bca56472d45112529efdba51c14e3fb65010ac5"} Jul 06 12:55:30.725022 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:30.724686 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:30.777261 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:30.777214 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" podStartSLOduration=2.152344585 podStartE2EDuration="6.777195779s" totalImagesPullingTime="3.542246226s" totalInitContainerRuntime="1.082604968s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:55:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:55:30.775871158 +0000 UTC m=+530.857312205" watchObservedRunningTime="2026-07-06 12:55:30.777195779 +0000 UTC m=+530.858636806" Jul 06 12:55:41.731510 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:41.731432 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58899bc45d-ts8m8" Jul 06 12:55:44.907788 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.907752 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc"] Jul 06 12:55:44.910906 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.910889 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:44.928104 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.928080 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-x9rtj\"" Jul 06 12:55:44.929058 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.929041 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 12:55:44.929177 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.929116 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 12:55:44.929177 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.929134 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 12:55:44.929177 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.929138 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 12:55:44.929616 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.929603 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 12:55:44.942641 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.942615 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc"] Jul 06 12:55:44.970576 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.970537 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9738dffd-b575-4842-bfde-f1a25e2c8b57-cert\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:44.970576 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.970574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvdtb\" (UniqueName: \"kubernetes.io/projected/9738dffd-b575-4842-bfde-f1a25e2c8b57-kube-api-access-rvdtb\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:44.970792 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.970597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9738dffd-b575-4842-bfde-f1a25e2c8b57-metrics-cert\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:44.970792 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:44.970661 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9738dffd-b575-4842-bfde-f1a25e2c8b57-manager-config\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.071923 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.071876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9738dffd-b575-4842-bfde-f1a25e2c8b57-cert\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.071923 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.071925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvdtb\" (UniqueName: \"kubernetes.io/projected/9738dffd-b575-4842-bfde-f1a25e2c8b57-kube-api-access-rvdtb\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.072233 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.071952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9738dffd-b575-4842-bfde-f1a25e2c8b57-metrics-cert\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.072233 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.072013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9738dffd-b575-4842-bfde-f1a25e2c8b57-manager-config\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.072700 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.072675 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9738dffd-b575-4842-bfde-f1a25e2c8b57-manager-config\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.074473 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.074453 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9738dffd-b575-4842-bfde-f1a25e2c8b57-cert\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.074533 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.074494 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9738dffd-b575-4842-bfde-f1a25e2c8b57-metrics-cert\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.104434 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.104408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvdtb\" (UniqueName: \"kubernetes.io/projected/9738dffd-b575-4842-bfde-f1a25e2c8b57-kube-api-access-rvdtb\") pod \"lws-controller-manager-67468f6576-qv2tc\" (UID: \"9738dffd-b575-4842-bfde-f1a25e2c8b57\") " pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.219535 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.219499 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:45.460558 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.460521 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc"] Jul 06 12:55:45.463654 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:55:45.463613 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9738dffd_b575_4842_bfde_f1a25e2c8b57.slice/crio-1daa58184508f6dea7de89bdf8f89f0640dcb9c89b79b86b64afcf72c0f3c75f WatchSource:0}: Error finding container 1daa58184508f6dea7de89bdf8f89f0640dcb9c89b79b86b64afcf72c0f3c75f: Status 404 returned error can't find the container with id 1daa58184508f6dea7de89bdf8f89f0640dcb9c89b79b86b64afcf72c0f3c75f Jul 06 12:55:45.788019 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:45.787984 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" event={"ID":"9738dffd-b575-4842-bfde-f1a25e2c8b57","Type":"ContainerStarted","Data":"1daa58184508f6dea7de89bdf8f89f0640dcb9c89b79b86b64afcf72c0f3c75f"} Jul 06 12:55:48.804645 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:48.804612 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" event={"ID":"9738dffd-b575-4842-bfde-f1a25e2c8b57","Type":"ContainerStarted","Data":"6f160934e775e558911e8809da2958d530d328acb648183a7ea66629b53300bd"} Jul 06 12:55:48.805027 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:48.804741 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:55:48.829247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:48.829196 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" podStartSLOduration=2.283962138 podStartE2EDuration="4.82917922s" totalImagesPullingTime="2.545217082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:55:48.827717801 +0000 UTC m=+548.909158855" watchObservedRunningTime="2026-07-06 12:55:48.82917922 +0000 UTC m=+548.910620236" Jul 06 12:55:55.580252 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.580211 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q"] Jul 06 12:55:55.582831 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.582808 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.596235 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:55.596210 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-136-21.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-21.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 06 12:55:55.596341 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:55.596266 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-tls\" is forbidden: User \"system:node:ip-10-0-136-21.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-21.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" Jul 06 12:55:55.596752 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:55.596736 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-21.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-21.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 06 12:55:55.596829 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:55.596808 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-7965d5bf57-8jf4q\" is forbidden: User \"system:node:ip-10-0-136-21.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-21.ec2.internal' and this object" podUID="99201dad-aaa5-45d7-831d-f804fb332f3c" pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.606998 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:55.606973 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-dockercfg-jnnjr\" is forbidden: User \"system:node:ip-10-0-136-21.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-21.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jnnjr\"" type="*v1.Secret" Jul 06 12:55:55.608008 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:55.607988 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-21.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-21.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 06 12:55:55.665849 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.665820 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q"] Jul 06 12:55:55.666266 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.666247 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99201dad-aaa5-45d7-831d-f804fb332f3c-tls-certs\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.666332 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.666299 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b49n8\" (UniqueName: \"kubernetes.io/projected/99201dad-aaa5-45d7-831d-f804fb332f3c-kube-api-access-b49n8\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.666371 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.666344 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99201dad-aaa5-45d7-831d-f804fb332f3c-tmp\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.767110 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.767072 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99201dad-aaa5-45d7-831d-f804fb332f3c-tls-certs\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.767309 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.767146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b49n8\" (UniqueName: \"kubernetes.io/projected/99201dad-aaa5-45d7-831d-f804fb332f3c-kube-api-access-b49n8\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.767309 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.767206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99201dad-aaa5-45d7-831d-f804fb332f3c-tmp\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:55.769573 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:55.769543 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99201dad-aaa5-45d7-831d-f804fb332f3c-tmp\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:56.427689 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:56.427660 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 12:55:56.543259 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:56.543228 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jnnjr\"" Jul 06 12:55:56.767546 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:56.767472 2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: failed to sync secret cache: timed out waiting for the condition Jul 06 12:55:56.767905 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:56.767562 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99201dad-aaa5-45d7-831d-f804fb332f3c-tls-certs podName:99201dad-aaa5-45d7-831d-f804fb332f3c nodeName:}" failed. No retries permitted until 2026-07-06 12:55:57.267544954 +0000 UTC m=+557.348985960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/99201dad-aaa5-45d7-831d-f804fb332f3c-tls-certs") pod "kube-auth-proxy-7965d5bf57-8jf4q" (UID: "99201dad-aaa5-45d7-831d-f804fb332f3c") : failed to sync secret cache: timed out waiting for the condition Jul 06 12:55:56.787140 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:56.787112 2559 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 06 12:55:57.093146 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.093055 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 12:55:57.112219 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.112194 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 12:55:57.155098 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.155070 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 12:55:57.158145 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:57.158128 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b49n8 for pod openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q: failed to sync configmap cache: timed out waiting for the condition Jul 06 12:55:57.158249 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:55:57.158212 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99201dad-aaa5-45d7-831d-f804fb332f3c-kube-api-access-b49n8 podName:99201dad-aaa5-45d7-831d-f804fb332f3c nodeName:}" failed. No retries permitted until 2026-07-06 12:55:57.658194126 +0000 UTC m=+557.739635132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b49n8" (UniqueName: "kubernetes.io/projected/99201dad-aaa5-45d7-831d-f804fb332f3c-kube-api-access-b49n8") pod "kube-auth-proxy-7965d5bf57-8jf4q" (UID: "99201dad-aaa5-45d7-831d-f804fb332f3c") : failed to sync configmap cache: timed out waiting for the condition Jul 06 12:55:57.282137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.282098 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99201dad-aaa5-45d7-831d-f804fb332f3c-tls-certs\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:57.284633 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.284610 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99201dad-aaa5-45d7-831d-f804fb332f3c-tls-certs\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:57.686225 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.686187 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b49n8\" (UniqueName: \"kubernetes.io/projected/99201dad-aaa5-45d7-831d-f804fb332f3c-kube-api-access-b49n8\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:57.688711 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.688687 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b49n8\" (UniqueName: \"kubernetes.io/projected/99201dad-aaa5-45d7-831d-f804fb332f3c-kube-api-access-b49n8\") pod \"kube-auth-proxy-7965d5bf57-8jf4q\" (UID: \"99201dad-aaa5-45d7-831d-f804fb332f3c\") " pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:57.691437 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.691421 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" Jul 06 12:55:57.895380 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:57.895350 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q"] Jul 06 12:55:57.898779 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:55:57.898754 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99201dad_aaa5_45d7_831d_f804fb332f3c.slice/crio-7e9dfccecd17477b3f992cd9447169aad4292d2cfad1dcac916ea2839f7d103f WatchSource:0}: Error finding container 7e9dfccecd17477b3f992cd9447169aad4292d2cfad1dcac916ea2839f7d103f: Status 404 returned error can't find the container with id 7e9dfccecd17477b3f992cd9447169aad4292d2cfad1dcac916ea2839f7d103f Jul 06 12:55:58.849323 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:58.849284 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" event={"ID":"99201dad-aaa5-45d7-831d-f804fb332f3c","Type":"ContainerStarted","Data":"7e9dfccecd17477b3f992cd9447169aad4292d2cfad1dcac916ea2839f7d103f"} Jul 06 12:55:59.811302 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:55:59.811272 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67468f6576-qv2tc" Jul 06 12:56:00.860584 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:56:00.860551 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" event={"ID":"99201dad-aaa5-45d7-831d-f804fb332f3c","Type":"ContainerStarted","Data":"4a12898153995108e12be0ae962bbee0f6f2ea899f59239dacdfada61320be81"} Jul 06 12:56:00.920424 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:56:00.920363 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7965d5bf57-8jf4q" podStartSLOduration=3.629196245 podStartE2EDuration="5.920348078s" totalImagesPullingTime="2.291151833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:56:00.919974507 +0000 UTC m=+561.001415578" watchObservedRunningTime="2026-07-06 12:56:00.920348078 +0000 UTC m=+561.001789105" Jul 06 12:56:40.370088 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:56:40.370059 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:56:40.372247 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:56:40.372223 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 12:56:40.374201 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:56:40.374179 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:56:40.376122 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:56:40.376106 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 12:57:27.424346 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.424309 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj"] Jul 06 12:57:27.426830 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.426814 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:57:27.430664 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.430643 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 12:57:27.431584 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.431564 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 12:57:27.431656 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.431592 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-w6p2x\"" Jul 06 12:57:27.458745 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.458717 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj"] Jul 06 12:57:27.494040 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.494000 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsxgh\" (UniqueName: \"kubernetes.io/projected/6a442a44-f29b-4748-bc96-2ff1195dfeee-kube-api-access-vsxgh\") pod \"authorino-operator-75cc8d5cb7-r9wbj\" (UID: \"6a442a44-f29b-4748-bc96-2ff1195dfeee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:57:27.595098 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.595056 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsxgh\" (UniqueName: \"kubernetes.io/projected/6a442a44-f29b-4748-bc96-2ff1195dfeee-kube-api-access-vsxgh\") pod \"authorino-operator-75cc8d5cb7-r9wbj\" (UID: \"6a442a44-f29b-4748-bc96-2ff1195dfeee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:57:27.620003 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.619974 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsxgh\" (UniqueName: \"kubernetes.io/projected/6a442a44-f29b-4748-bc96-2ff1195dfeee-kube-api-access-vsxgh\") pod \"authorino-operator-75cc8d5cb7-r9wbj\" (UID: \"6a442a44-f29b-4748-bc96-2ff1195dfeee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:57:27.737098 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.737008 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:57:27.909443 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:27.909410 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj"] Jul 06 12:57:27.912731 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:57:27.912703 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a442a44_f29b_4748_bc96_2ff1195dfeee.slice/crio-29cb0030680ebcb2ea5e195aa7983f9d77b1130a7f73564019dde8d03fe8f4cc WatchSource:0}: Error finding container 29cb0030680ebcb2ea5e195aa7983f9d77b1130a7f73564019dde8d03fe8f4cc: Status 404 returned error can't find the container with id 29cb0030680ebcb2ea5e195aa7983f9d77b1130a7f73564019dde8d03fe8f4cc Jul 06 12:57:28.246970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:28.246931 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" event={"ID":"6a442a44-f29b-4748-bc96-2ff1195dfeee","Type":"ContainerStarted","Data":"29cb0030680ebcb2ea5e195aa7983f9d77b1130a7f73564019dde8d03fe8f4cc"} Jul 06 12:57:31.265011 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:31.264974 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" event={"ID":"6a442a44-f29b-4748-bc96-2ff1195dfeee","Type":"ContainerStarted","Data":"677f03e4e02432149f8facbbd7c0f126d5c61819cea52282161dd0fd449c2495"} Jul 06 12:57:31.265505 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:31.265149 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:57:31.325501 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:31.325445 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" podStartSLOduration=1.7448579039999998 podStartE2EDuration="4.325427814s" totalImagesPullingTime="2.58056991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:57:31.32514406 +0000 UTC m=+651.406585108" watchObservedRunningTime="2026-07-06 12:57:31.325427814 +0000 UTC m=+651.406868841" Jul 06 12:57:42.271574 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:57:42.271543 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r9wbj" Jul 06 12:58:26.364348 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.364314 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cnpdx"] Jul 06 12:58:26.367622 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.367605 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:26.370288 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.370268 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j7wql\"" Jul 06 12:58:26.377359 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.377338 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cnpdx"] Jul 06 12:58:26.526002 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.525963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptcgk\" (UniqueName: \"kubernetes.io/projected/1f426306-53e9-4813-8333-8b24a0cf5e6d-kube-api-access-ptcgk\") pod \"authorino-58cd955675-cnpdx\" (UID: \"1f426306-53e9-4813-8333-8b24a0cf5e6d\") " pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:26.600039 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.600002 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-k2qzc"] Jul 06 12:58:26.603355 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.603337 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:58:26.613554 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.613526 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k2qzc"] Jul 06 12:58:26.627054 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.626981 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptcgk\" (UniqueName: \"kubernetes.io/projected/1f426306-53e9-4813-8333-8b24a0cf5e6d-kube-api-access-ptcgk\") pod \"authorino-58cd955675-cnpdx\" (UID: \"1f426306-53e9-4813-8333-8b24a0cf5e6d\") " pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:26.635696 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.635675 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptcgk\" (UniqueName: \"kubernetes.io/projected/1f426306-53e9-4813-8333-8b24a0cf5e6d-kube-api-access-ptcgk\") pod \"authorino-58cd955675-cnpdx\" (UID: \"1f426306-53e9-4813-8333-8b24a0cf5e6d\") " pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:26.676575 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.676539 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:26.727970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.727936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppl4s\" (UniqueName: \"kubernetes.io/projected/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16-kube-api-access-ppl4s\") pod \"authorino-75d556bd77-k2qzc\" (UID: \"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16\") " pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:58:26.807277 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.807250 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cnpdx"] Jul 06 12:58:26.809007 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:58:26.808979 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f426306_53e9_4813_8333_8b24a0cf5e6d.slice/crio-d2bf79aec3adb7d40cefa00ebfcd410af02c6e6310609954d6398284a6ad8519 WatchSource:0}: Error finding container d2bf79aec3adb7d40cefa00ebfcd410af02c6e6310609954d6398284a6ad8519: Status 404 returned error can't find the container with id d2bf79aec3adb7d40cefa00ebfcd410af02c6e6310609954d6398284a6ad8519 Jul 06 12:58:26.828698 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.828669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppl4s\" (UniqueName: \"kubernetes.io/projected/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16-kube-api-access-ppl4s\") pod \"authorino-75d556bd77-k2qzc\" (UID: \"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16\") " pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:58:26.838183 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.838135 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppl4s\" (UniqueName: \"kubernetes.io/projected/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16-kube-api-access-ppl4s\") pod \"authorino-75d556bd77-k2qzc\" (UID: \"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16\") " pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:58:26.912510 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:26.912437 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:58:27.049220 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:27.049195 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k2qzc"] Jul 06 12:58:27.051042 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:58:27.051020 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f9b36b0_5bab_46d3_9ffa_1fd342f86a16.slice/crio-1a2489fce0fb6e69a9a9aac71256874e83f1074bb68f63b0820f99148bde2b60 WatchSource:0}: Error finding container 1a2489fce0fb6e69a9a9aac71256874e83f1074bb68f63b0820f99148bde2b60: Status 404 returned error can't find the container with id 1a2489fce0fb6e69a9a9aac71256874e83f1074bb68f63b0820f99148bde2b60 Jul 06 12:58:27.512506 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:27.512446 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k2qzc" event={"ID":"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16","Type":"ContainerStarted","Data":"1a2489fce0fb6e69a9a9aac71256874e83f1074bb68f63b0820f99148bde2b60"} Jul 06 12:58:27.515058 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:27.515013 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cnpdx" event={"ID":"1f426306-53e9-4813-8333-8b24a0cf5e6d","Type":"ContainerStarted","Data":"d2bf79aec3adb7d40cefa00ebfcd410af02c6e6310609954d6398284a6ad8519"} Jul 06 12:58:30.535993 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:30.535948 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cnpdx" event={"ID":"1f426306-53e9-4813-8333-8b24a0cf5e6d","Type":"ContainerStarted","Data":"b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677"} Jul 06 12:58:30.538391 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:30.538362 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k2qzc" event={"ID":"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16","Type":"ContainerStarted","Data":"edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023"} Jul 06 12:58:30.582486 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:30.582424 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-k2qzc" podStartSLOduration=1.2798943010000001 podStartE2EDuration="4.582403606s" totalImagesPullingTime="3.302509305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:58:30.580904581 +0000 UTC m=+710.662345618" watchObservedRunningTime="2026-07-06 12:58:30.582403606 +0000 UTC m=+710.663844635" Jul 06 12:58:30.582668 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:30.582614 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cnpdx" podStartSLOduration=1.026460355 podStartE2EDuration="4.582606037s" totalImagesPullingTime="3.556145682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:58:30.557923085 +0000 UTC m=+710.639364112" watchObservedRunningTime="2026-07-06 12:58:30.582606037 +0000 UTC m=+710.664047064" Jul 06 12:58:30.632338 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:30.632261 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cnpdx"] Jul 06 12:58:32.547574 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:32.547532 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cnpdx" podUID="1f426306-53e9-4813-8333-8b24a0cf5e6d" containerName="authorino" containerID="cri-o://b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677" gracePeriod=30 Jul 06 12:58:32.804873 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:32.804811 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:32.885314 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:32.885279 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptcgk\" (UniqueName: \"kubernetes.io/projected/1f426306-53e9-4813-8333-8b24a0cf5e6d-kube-api-access-ptcgk\") pod \"1f426306-53e9-4813-8333-8b24a0cf5e6d\" (UID: \"1f426306-53e9-4813-8333-8b24a0cf5e6d\") " Jul 06 12:58:32.887472 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:32.887437 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f426306-53e9-4813-8333-8b24a0cf5e6d-kube-api-access-ptcgk" (OuterVolumeSpecName: "kube-api-access-ptcgk") pod "1f426306-53e9-4813-8333-8b24a0cf5e6d" (UID: "1f426306-53e9-4813-8333-8b24a0cf5e6d"). InnerVolumeSpecName "kube-api-access-ptcgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:58:32.986599 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:32.986549 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptcgk\" (UniqueName: \"kubernetes.io/projected/1f426306-53e9-4813-8333-8b24a0cf5e6d-kube-api-access-ptcgk\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:58:33.552940 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.552901 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f426306-53e9-4813-8333-8b24a0cf5e6d" containerID="b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677" exitCode=0 Jul 06 12:58:33.553418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.552954 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cnpdx" Jul 06 12:58:33.553418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.552995 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cnpdx" event={"ID":"1f426306-53e9-4813-8333-8b24a0cf5e6d","Type":"ContainerDied","Data":"b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677"} Jul 06 12:58:33.553418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.553047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cnpdx" event={"ID":"1f426306-53e9-4813-8333-8b24a0cf5e6d","Type":"ContainerDied","Data":"d2bf79aec3adb7d40cefa00ebfcd410af02c6e6310609954d6398284a6ad8519"} Jul 06 12:58:33.553418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.553072 2559 scope.go:117] "RemoveContainer" containerID="b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677" Jul 06 12:58:33.561331 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.561314 2559 scope.go:117] "RemoveContainer" containerID="b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677" Jul 06 12:58:33.561600 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:58:33.561582 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677\": container with ID starting with b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677 not found: ID does not exist" containerID="b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677" Jul 06 12:58:33.561646 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.561609 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677"} err="failed to get container status \"b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677\": rpc error: code = NotFound desc = could not find container \"b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677\": container with ID starting with b26bfc4acda65b8b9996321916dc6c80a0d41badf8ca3e67958d6d4a6c933677 not found: ID does not exist" Jul 06 12:58:33.577212 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.577184 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cnpdx"] Jul 06 12:58:33.581797 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:33.581777 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cnpdx"] Jul 06 12:58:34.415448 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:58:34.415402 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f426306-53e9-4813-8333-8b24a0cf5e6d" path="/var/lib/kubelet/pods/1f426306-53e9-4813-8333-8b24a0cf5e6d/volumes" Jul 06 12:59:00.291221 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.291184 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sr8nz"] Jul 06 12:59:00.291800 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.291693 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f426306-53e9-4813-8333-8b24a0cf5e6d" containerName="authorino" Jul 06 12:59:00.291800 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.291723 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f426306-53e9-4813-8333-8b24a0cf5e6d" containerName="authorino" Jul 06 12:59:00.291932 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.291820 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f426306-53e9-4813-8333-8b24a0cf5e6d" containerName="authorino" Jul 06 12:59:00.295613 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.295591 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:00.302558 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.302523 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sr8nz"] Jul 06 12:59:00.428654 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.428626 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6lcm\" (UniqueName: \"kubernetes.io/projected/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4-kube-api-access-g6lcm\") pod \"authorino-5d4b67c446-sr8nz\" (UID: \"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4\") " pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:00.529502 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.529473 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6lcm\" (UniqueName: \"kubernetes.io/projected/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4-kube-api-access-g6lcm\") pod \"authorino-5d4b67c446-sr8nz\" (UID: \"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4\") " pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:00.542475 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.542418 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6lcm\" (UniqueName: \"kubernetes.io/projected/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4-kube-api-access-g6lcm\") pod \"authorino-5d4b67c446-sr8nz\" (UID: \"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4\") " pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:00.567783 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.567752 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sr8nz"] Jul 06 12:59:00.568025 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.568010 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:00.593101 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.593073 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6948cd47f-7xlgd"] Jul 06 12:59:00.595284 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.595267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:00.605399 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.605378 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6948cd47f-7xlgd"] Jul 06 12:59:00.656798 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.656742 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6948cd47f-7xlgd"] Jul 06 12:59:00.657542 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:00.657346 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-85226], unattached volumes=[], failed to process volumes=[kube-api-access-85226]: context canceled" pod="kuadrant-system/authorino-6948cd47f-7xlgd" podUID="ee9a941a-5ef3-4c8d-a28b-36cb63e05d88" Jul 06 12:59:00.674145 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.674122 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:00.680782 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.680760 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:00.698921 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.698897 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bcfccb499-dvmz2"] Jul 06 12:59:00.701105 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.701076 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:00.704296 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.703817 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 12:59:00.709420 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.709348 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sr8nz"] Jul 06 12:59:00.711067 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.711046 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bcfccb499-dvmz2"] Jul 06 12:59:00.711766 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:59:00.711627 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44c2e20f_d7df_4c20_a3ea_f4876e08d6b4.slice/crio-31f5904a21642049c01a2b6ae56c68eb0ca3d5d022bac296019257085cbdeadc WatchSource:0}: Error finding container 31f5904a21642049c01a2b6ae56c68eb0ca3d5d022bac296019257085cbdeadc: Status 404 returned error can't find the container with id 31f5904a21642049c01a2b6ae56c68eb0ca3d5d022bac296019257085cbdeadc Jul 06 12:59:00.730990 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.730967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85226\" (UniqueName: \"kubernetes.io/projected/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88-kube-api-access-85226\") pod \"authorino-6948cd47f-7xlgd\" (UID: \"ee9a941a-5ef3-4c8d-a28b-36cb63e05d88\") " pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:00.832317 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.832251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f436abd7-3602-41ec-b4ed-35877a49a738-tls-cert\") pod \"authorino-5bcfccb499-dvmz2\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:00.832317 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.832294 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85226\" (UniqueName: \"kubernetes.io/projected/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88-kube-api-access-85226\") pod \"authorino-6948cd47f-7xlgd\" (UID: \"ee9a941a-5ef3-4c8d-a28b-36cb63e05d88\") " pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:00.832476 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.832331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz8ck\" (UniqueName: \"kubernetes.io/projected/f436abd7-3602-41ec-b4ed-35877a49a738-kube-api-access-rz8ck\") pod \"authorino-5bcfccb499-dvmz2\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:00.841331 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.841303 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85226\" (UniqueName: \"kubernetes.io/projected/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88-kube-api-access-85226\") pod \"authorino-6948cd47f-7xlgd\" (UID: \"ee9a941a-5ef3-4c8d-a28b-36cb63e05d88\") " pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:00.933004 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.932980 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85226\" (UniqueName: \"kubernetes.io/projected/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88-kube-api-access-85226\") pod \"ee9a941a-5ef3-4c8d-a28b-36cb63e05d88\" (UID: \"ee9a941a-5ef3-4c8d-a28b-36cb63e05d88\") " Jul 06 12:59:00.933137 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.933124 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f436abd7-3602-41ec-b4ed-35877a49a738-tls-cert\") pod \"authorino-5bcfccb499-dvmz2\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:00.933211 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.933180 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz8ck\" (UniqueName: \"kubernetes.io/projected/f436abd7-3602-41ec-b4ed-35877a49a738-kube-api-access-rz8ck\") pod \"authorino-5bcfccb499-dvmz2\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:00.934874 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.934846 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88-kube-api-access-85226" (OuterVolumeSpecName: "kube-api-access-85226") pod "ee9a941a-5ef3-4c8d-a28b-36cb63e05d88" (UID: "ee9a941a-5ef3-4c8d-a28b-36cb63e05d88"). InnerVolumeSpecName "kube-api-access-85226". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:59:00.935374 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.935354 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f436abd7-3602-41ec-b4ed-35877a49a738-tls-cert\") pod \"authorino-5bcfccb499-dvmz2\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:00.941490 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:00.941472 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz8ck\" (UniqueName: \"kubernetes.io/projected/f436abd7-3602-41ec-b4ed-35877a49a738-kube-api-access-rz8ck\") pod \"authorino-5bcfccb499-dvmz2\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:01.012235 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.012216 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 12:59:01.033638 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.033621 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85226\" (UniqueName: \"kubernetes.io/projected/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88-kube-api-access-85226\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:59:01.157450 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.157298 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bcfccb499-dvmz2"] Jul 06 12:59:01.159917 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:59:01.159891 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf436abd7_3602_41ec_b4ed_35877a49a738.slice/crio-bc5064e01eb37e4d17e8a587eb8c717739a77f3f17fb53620cad1637eea1f83a WatchSource:0}: Error finding container bc5064e01eb37e4d17e8a587eb8c717739a77f3f17fb53620cad1637eea1f83a: Status 404 returned error can't find the container with id bc5064e01eb37e4d17e8a587eb8c717739a77f3f17fb53620cad1637eea1f83a Jul 06 12:59:01.681798 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.681756 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" event={"ID":"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4","Type":"ContainerStarted","Data":"131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852"} Jul 06 12:59:01.682408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.681810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" event={"ID":"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4","Type":"ContainerStarted","Data":"31f5904a21642049c01a2b6ae56c68eb0ca3d5d022bac296019257085cbdeadc"} Jul 06 12:59:01.682408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.681820 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" podUID="44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" containerName="authorino" containerID="cri-o://131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852" gracePeriod=30 Jul 06 12:59:01.683621 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.683573 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" event={"ID":"f436abd7-3602-41ec-b4ed-35877a49a738","Type":"ContainerStarted","Data":"61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6"} Jul 06 12:59:01.683621 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.683601 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" event={"ID":"f436abd7-3602-41ec-b4ed-35877a49a738","Type":"ContainerStarted","Data":"bc5064e01eb37e4d17e8a587eb8c717739a77f3f17fb53620cad1637eea1f83a"} Jul 06 12:59:01.683621 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.683607 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6948cd47f-7xlgd" Jul 06 12:59:01.698412 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.698368 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" podStartSLOduration=1.423892448 podStartE2EDuration="1.698352064s" totalImagesPullingTime="274.459616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:59:01.696831251 +0000 UTC m=+741.778272304" watchObservedRunningTime="2026-07-06 12:59:01.698352064 +0000 UTC m=+741.779793093" Jul 06 12:59:01.719752 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.719700 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" podStartSLOduration=1.342276378 podStartE2EDuration="1.719685873s" totalImagesPullingTime="377.409495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:59:01.718255936 +0000 UTC m=+741.799696962" watchObservedRunningTime="2026-07-06 12:59:01.719685873 +0000 UTC m=+741.801126900" Jul 06 12:59:01.750260 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.750222 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k2qzc"] Jul 06 12:59:01.750429 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.750395 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-k2qzc" podUID="1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" containerName="authorino" containerID="cri-o://edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023" gracePeriod=30 Jul 06 12:59:01.791967 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.791900 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6948cd47f-7xlgd"] Jul 06 12:59:01.796304 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:01.796280 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6948cd47f-7xlgd"] Jul 06 12:59:02.002462 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.002442 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:02.005407 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.005383 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:59:02.144394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.144312 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppl4s\" (UniqueName: \"kubernetes.io/projected/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16-kube-api-access-ppl4s\") pod \"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16\" (UID: \"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16\") " Jul 06 12:59:02.144394 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.144362 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6lcm\" (UniqueName: \"kubernetes.io/projected/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4-kube-api-access-g6lcm\") pod \"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4\" (UID: \"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4\") " Jul 06 12:59:02.146607 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.146577 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4-kube-api-access-g6lcm" (OuterVolumeSpecName: "kube-api-access-g6lcm") pod "44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" (UID: "44c2e20f-d7df-4c20-a3ea-f4876e08d6b4"). InnerVolumeSpecName "kube-api-access-g6lcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:59:02.146690 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.146582 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16-kube-api-access-ppl4s" (OuterVolumeSpecName: "kube-api-access-ppl4s") pod "1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" (UID: "1f9b36b0-5bab-46d3-9ffa-1fd342f86a16"). InnerVolumeSpecName "kube-api-access-ppl4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:59:02.245403 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.245370 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppl4s\" (UniqueName: \"kubernetes.io/projected/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16-kube-api-access-ppl4s\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:59:02.245403 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.245396 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6lcm\" (UniqueName: \"kubernetes.io/projected/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4-kube-api-access-g6lcm\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:59:02.416289 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.416197 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9a941a-5ef3-4c8d-a28b-36cb63e05d88" path="/var/lib/kubelet/pods/ee9a941a-5ef3-4c8d-a28b-36cb63e05d88/volumes" Jul 06 12:59:02.689666 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.689635 2559 generic.go:356] "Generic (PLEG): container finished" podID="44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" containerID="131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852" exitCode=0 Jul 06 12:59:02.690088 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.689677 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" Jul 06 12:59:02.690088 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.689719 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" event={"ID":"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4","Type":"ContainerDied","Data":"131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852"} Jul 06 12:59:02.690088 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.689756 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-sr8nz" event={"ID":"44c2e20f-d7df-4c20-a3ea-f4876e08d6b4","Type":"ContainerDied","Data":"31f5904a21642049c01a2b6ae56c68eb0ca3d5d022bac296019257085cbdeadc"} Jul 06 12:59:02.690088 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.689776 2559 scope.go:117] "RemoveContainer" containerID="131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852" Jul 06 12:59:02.691156 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.691136 2559 generic.go:356] "Generic (PLEG): container finished" podID="1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" containerID="edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023" exitCode=0 Jul 06 12:59:02.691224 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.691197 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k2qzc" Jul 06 12:59:02.691257 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.691225 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k2qzc" event={"ID":"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16","Type":"ContainerDied","Data":"edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023"} Jul 06 12:59:02.691257 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.691253 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k2qzc" event={"ID":"1f9b36b0-5bab-46d3-9ffa-1fd342f86a16","Type":"ContainerDied","Data":"1a2489fce0fb6e69a9a9aac71256874e83f1074bb68f63b0820f99148bde2b60"} Jul 06 12:59:02.698349 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.698333 2559 scope.go:117] "RemoveContainer" containerID="131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852" Jul 06 12:59:02.698589 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:02.698571 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852\": container with ID starting with 131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852 not found: ID does not exist" containerID="131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852" Jul 06 12:59:02.698630 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.698601 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852"} err="failed to get container status \"131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852\": rpc error: code = NotFound desc = could not find container \"131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852\": container with ID starting with 131aab9069928086f0ec995b7483cfa4eb8b9106802a41cf4b8cc6e99ce14852 not found: ID does not exist" Jul 06 12:59:02.698630 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.698618 2559 scope.go:117] "RemoveContainer" containerID="edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023" Jul 06 12:59:02.705866 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.705844 2559 scope.go:117] "RemoveContainer" containerID="edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023" Jul 06 12:59:02.706084 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:02.706067 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023\": container with ID starting with edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023 not found: ID does not exist" containerID="edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023" Jul 06 12:59:02.706129 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.706094 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023"} err="failed to get container status \"edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023\": rpc error: code = NotFound desc = could not find container \"edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023\": container with ID starting with edaa26a8e0387b31a6596e2003bcc4d99ea0cbf9c7e931b29341ff3051e51023 not found: ID does not exist" Jul 06 12:59:02.726673 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.726642 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sr8nz"] Jul 06 12:59:02.732302 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.732281 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-sr8nz"] Jul 06 12:59:02.775971 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.775946 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k2qzc"] Jul 06 12:59:02.785906 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:02.785881 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k2qzc"] Jul 06 12:59:04.415633 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:04.415600 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" path="/var/lib/kubelet/pods/1f9b36b0-5bab-46d3-9ffa-1fd342f86a16/volumes" Jul 06 12:59:04.416002 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:04.415919 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" path="/var/lib/kubelet/pods/44c2e20f-d7df-4c20-a3ea-f4876e08d6b4/volumes" Jul 06 12:59:06.301031 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.300997 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64cc99fdbc-dxdsq"] Jul 06 12:59:06.301493 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.301400 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" containerName="authorino" Jul 06 12:59:06.301493 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.301412 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" containerName="authorino" Jul 06 12:59:06.301493 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.301435 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" containerName="authorino" Jul 06 12:59:06.301493 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.301440 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" containerName="authorino" Jul 06 12:59:06.301622 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.301499 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9b36b0-5bab-46d3-9ffa-1fd342f86a16" containerName="authorino" Jul 06 12:59:06.301622 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.301507 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="44c2e20f-d7df-4c20-a3ea-f4876e08d6b4" containerName="authorino" Jul 06 12:59:06.303378 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.303362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.306040 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.306019 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 12:59:06.306403 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.306386 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 12:59:06.306504 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.306491 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2tqdb\"" Jul 06 12:59:06.323305 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.323283 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64cc99fdbc-dxdsq"] Jul 06 12:59:06.381811 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.381786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.381946 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.381826 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcqh2\" (UniqueName: \"kubernetes.io/projected/6562bf72-10aa-4aea-8263-b185c3e3c05c-kube-api-access-dcqh2\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.482409 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.482382 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.482591 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.482428 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcqh2\" (UniqueName: \"kubernetes.io/projected/6562bf72-10aa-4aea-8263-b185c3e3c05c-kube-api-access-dcqh2\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.482591 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:06.482520 2559 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 12:59:06.482591 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:06.482584 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert podName:6562bf72-10aa-4aea-8263-b185c3e3c05c nodeName:}" failed. No retries permitted until 2026-07-06 12:59:06.982563797 +0000 UTC m=+747.064004807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert") pod "maas-controller-64cc99fdbc-dxdsq" (UID: "6562bf72-10aa-4aea-8263-b185c3e3c05c") : secret "maas-controller-webhook-cert" not found Jul 06 12:59:06.488966 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.488941 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-775867fb56-sxsjp"] Jul 06 12:59:06.491275 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.491260 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:06.502927 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.502895 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775867fb56-sxsjp"] Jul 06 12:59:06.503408 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.503390 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcqh2\" (UniqueName: \"kubernetes.io/projected/6562bf72-10aa-4aea-8263-b185c3e3c05c-kube-api-access-dcqh2\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.583257 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.583156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:06.583257 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.583212 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7s2g\" (UniqueName: \"kubernetes.io/projected/d142d295-5165-4de9-938e-fd526540d6af-kube-api-access-l7s2g\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:06.684614 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.684586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:06.684758 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.684624 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7s2g\" (UniqueName: \"kubernetes.io/projected/d142d295-5165-4de9-938e-fd526540d6af-kube-api-access-l7s2g\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:06.684758 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:06.684739 2559 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 12:59:06.684834 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:06.684817 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert podName:d142d295-5165-4de9-938e-fd526540d6af nodeName:}" failed. No retries permitted until 2026-07-06 12:59:07.184801053 +0000 UTC m=+747.266242063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert") pod "maas-controller-775867fb56-sxsjp" (UID: "d142d295-5165-4de9-938e-fd526540d6af") : secret "maas-controller-webhook-cert" not found Jul 06 12:59:06.696079 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.696055 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7s2g\" (UniqueName: \"kubernetes.io/projected/d142d295-5165-4de9-938e-fd526540d6af-kube-api-access-l7s2g\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:06.987654 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.987619 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:06.989959 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:06.989940 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert\") pod \"maas-controller-64cc99fdbc-dxdsq\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:07.190097 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.190062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:07.192484 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.192453 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert\") pod \"maas-controller-775867fb56-sxsjp\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:07.213312 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.213288 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:07.343182 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.343130 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64cc99fdbc-dxdsq"] Jul 06 12:59:07.347142 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:59:07.347117 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6562bf72_10aa_4aea_8263_b185c3e3c05c.slice/crio-106468c1ed08f720058ab1a4b8c9e4ad35840ea27b60007838d85032ade3ff40 WatchSource:0}: Error finding container 106468c1ed08f720058ab1a4b8c9e4ad35840ea27b60007838d85032ade3ff40: Status 404 returned error can't find the container with id 106468c1ed08f720058ab1a4b8c9e4ad35840ea27b60007838d85032ade3ff40 Jul 06 12:59:07.400719 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.400695 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:07.528296 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.528146 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775867fb56-sxsjp"] Jul 06 12:59:07.530396 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:59:07.530368 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd142d295_5165_4de9_938e_fd526540d6af.slice/crio-ef3d73ba97c69609f0b9ae637720dfa767d79b1ff10641a59040dd2b9cdfd9d8 WatchSource:0}: Error finding container ef3d73ba97c69609f0b9ae637720dfa767d79b1ff10641a59040dd2b9cdfd9d8: Status 404 returned error can't find the container with id ef3d73ba97c69609f0b9ae637720dfa767d79b1ff10641a59040dd2b9cdfd9d8 Jul 06 12:59:07.717502 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.717466 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775867fb56-sxsjp" event={"ID":"d142d295-5165-4de9-938e-fd526540d6af","Type":"ContainerStarted","Data":"ef3d73ba97c69609f0b9ae637720dfa767d79b1ff10641a59040dd2b9cdfd9d8"} Jul 06 12:59:07.718520 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:07.718502 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" event={"ID":"6562bf72-10aa-4aea-8263-b185c3e3c05c","Type":"ContainerStarted","Data":"106468c1ed08f720058ab1a4b8c9e4ad35840ea27b60007838d85032ade3ff40"} Jul 06 12:59:09.732937 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:09.732848 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775867fb56-sxsjp" event={"ID":"d142d295-5165-4de9-938e-fd526540d6af","Type":"ContainerStarted","Data":"cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff"} Jul 06 12:59:09.733392 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:09.732954 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:09.734437 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:09.734414 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" event={"ID":"6562bf72-10aa-4aea-8263-b185c3e3c05c","Type":"ContainerStarted","Data":"30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5"} Jul 06 12:59:09.734558 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:09.734461 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:09.767953 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:09.767905 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-775867fb56-sxsjp" podStartSLOduration=1.851996601 podStartE2EDuration="3.767891511s" totalImagesPullingTime="1.91589491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:59:09.766355169 +0000 UTC m=+749.847796196" watchObservedRunningTime="2026-07-06 12:59:09.767891511 +0000 UTC m=+749.849332538" Jul 06 12:59:09.811812 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:09.811761 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" podStartSLOduration=1.7217611499999999 podStartE2EDuration="3.811748505s" totalImagesPullingTime="2.089987355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:59:09.808414548 +0000 UTC m=+749.889855577" watchObservedRunningTime="2026-07-06 12:59:09.811748505 +0000 UTC m=+749.893189531" Jul 06 12:59:20.746299 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:20.746262 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:20.746835 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:20.746816 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 12:59:20.814224 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:20.814159 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64cc99fdbc-dxdsq"] Jul 06 12:59:20.814438 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:20.814391 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" podUID="6562bf72-10aa-4aea-8263-b185c3e3c05c" containerName="manager" containerID="cri-o://30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5" gracePeriod=30 Jul 06 12:59:21.060964 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.060933 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:21.116439 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.116402 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert\") pod \"6562bf72-10aa-4aea-8263-b185c3e3c05c\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " Jul 06 12:59:21.116634 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.116463 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcqh2\" (UniqueName: \"kubernetes.io/projected/6562bf72-10aa-4aea-8263-b185c3e3c05c-kube-api-access-dcqh2\") pod \"6562bf72-10aa-4aea-8263-b185c3e3c05c\" (UID: \"6562bf72-10aa-4aea-8263-b185c3e3c05c\") " Jul 06 12:59:21.118681 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.118648 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6562bf72-10aa-4aea-8263-b185c3e3c05c-kube-api-access-dcqh2" (OuterVolumeSpecName: "kube-api-access-dcqh2") pod "6562bf72-10aa-4aea-8263-b185c3e3c05c" (UID: "6562bf72-10aa-4aea-8263-b185c3e3c05c"). InnerVolumeSpecName "kube-api-access-dcqh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 12:59:21.118681 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.118653 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6562bf72-10aa-4aea-8263-b185c3e3c05c" (UID: "6562bf72-10aa-4aea-8263-b185c3e3c05c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 12:59:21.217970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.217891 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcqh2\" (UniqueName: \"kubernetes.io/projected/6562bf72-10aa-4aea-8263-b185c3e3c05c-kube-api-access-dcqh2\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:59:21.217970 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.217921 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6562bf72-10aa-4aea-8263-b185c3e3c05c-webhook-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 12:59:21.797849 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.797817 2559 generic.go:356] "Generic (PLEG): container finished" podID="6562bf72-10aa-4aea-8263-b185c3e3c05c" containerID="30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5" exitCode=0 Jul 06 12:59:21.798357 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.797865 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" Jul 06 12:59:21.798357 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.797883 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" event={"ID":"6562bf72-10aa-4aea-8263-b185c3e3c05c","Type":"ContainerDied","Data":"30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5"} Jul 06 12:59:21.798357 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.797935 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64cc99fdbc-dxdsq" event={"ID":"6562bf72-10aa-4aea-8263-b185c3e3c05c","Type":"ContainerDied","Data":"106468c1ed08f720058ab1a4b8c9e4ad35840ea27b60007838d85032ade3ff40"} Jul 06 12:59:21.798357 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.797968 2559 scope.go:117] "RemoveContainer" containerID="30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5" Jul 06 12:59:21.807043 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.807016 2559 scope.go:117] "RemoveContainer" containerID="30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5" Jul 06 12:59:21.807313 ip-10-0-136-21 kubenswrapper[2559]: E0706 12:59:21.807294 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5\": container with ID starting with 30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5 not found: ID does not exist" containerID="30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5" Jul 06 12:59:21.807373 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.807321 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5"} err="failed to get container status \"30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5\": rpc error: code = NotFound desc = could not find container \"30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5\": container with ID starting with 30cc59c23a81e97d11109f5c8f259bb4370c183c2402aa3c8c94823f2acfcfd5 not found: ID does not exist" Jul 06 12:59:21.824258 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.824227 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64cc99fdbc-dxdsq"] Jul 06 12:59:21.830572 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:21.830548 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64cc99fdbc-dxdsq"] Jul 06 12:59:22.415991 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:22.415961 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6562bf72-10aa-4aea-8263-b185c3e3c05c" path="/var/lib/kubelet/pods/6562bf72-10aa-4aea-8263-b185c3e3c05c/volumes" Jul 06 12:59:45.004695 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.004654 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7c7f5bd644-7dc2h"] Jul 06 12:59:45.005270 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.005202 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6562bf72-10aa-4aea-8263-b185c3e3c05c" containerName="manager" Jul 06 12:59:45.005270 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.005218 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6562bf72-10aa-4aea-8263-b185c3e3c05c" containerName="manager" Jul 06 12:59:45.005418 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.005302 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6562bf72-10aa-4aea-8263-b185c3e3c05c" containerName="manager" Jul 06 12:59:45.010807 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.010786 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.013582 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.013559 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 12:59:45.013713 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.013559 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5pfrb\"" Jul 06 12:59:45.020653 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.020627 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7c7f5bd644-7dc2h"] Jul 06 12:59:45.131680 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.131637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44400754-fe37-4084-a232-b78c08b77b84-maas-api-tls\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.131871 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.131690 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44400754-fe37-4084-a232-b78c08b77b84-openshift-service-ca\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.131871 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.131818 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7kkh\" (UniqueName: \"kubernetes.io/projected/44400754-fe37-4084-a232-b78c08b77b84-kube-api-access-n7kkh\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.232576 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.232543 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7kkh\" (UniqueName: \"kubernetes.io/projected/44400754-fe37-4084-a232-b78c08b77b84-kube-api-access-n7kkh\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.232749 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.232623 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44400754-fe37-4084-a232-b78c08b77b84-maas-api-tls\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.232749 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.232650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44400754-fe37-4084-a232-b78c08b77b84-openshift-service-ca\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.233311 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.233291 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44400754-fe37-4084-a232-b78c08b77b84-openshift-service-ca\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.235113 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.235090 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44400754-fe37-4084-a232-b78c08b77b84-maas-api-tls\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.241547 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.241520 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7kkh\" (UniqueName: \"kubernetes.io/projected/44400754-fe37-4084-a232-b78c08b77b84-kube-api-access-n7kkh\") pod \"maas-api-7c7f5bd644-7dc2h\" (UID: \"44400754-fe37-4084-a232-b78c08b77b84\") " pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.322654 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.322556 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:45.465414 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.465387 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7c7f5bd644-7dc2h"] Jul 06 12:59:45.468088 ip-10-0-136-21 kubenswrapper[2559]: W0706 12:59:45.468055 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44400754_fe37_4084_a232_b78c08b77b84.slice/crio-64f49406882b2d6896c71c566fcef415bb63895b119cf6b3dbcc1a2d64272f43 WatchSource:0}: Error finding container 64f49406882b2d6896c71c566fcef415bb63895b119cf6b3dbcc1a2d64272f43: Status 404 returned error can't find the container with id 64f49406882b2d6896c71c566fcef415bb63895b119cf6b3dbcc1a2d64272f43 Jul 06 12:59:45.909074 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:45.909042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" event={"ID":"44400754-fe37-4084-a232-b78c08b77b84","Type":"ContainerStarted","Data":"64f49406882b2d6896c71c566fcef415bb63895b119cf6b3dbcc1a2d64272f43"} Jul 06 12:59:47.919290 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:47.919255 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" event={"ID":"44400754-fe37-4084-a232-b78c08b77b84","Type":"ContainerStarted","Data":"6aa80488c817e3892031aaa761b2ad9342be376ef4b7fb699ad6d0de11c7b201"} Jul 06 12:59:47.919684 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:47.919384 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 12:59:47.937854 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:47.937791 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" podStartSLOduration=2.462220347 podStartE2EDuration="3.937771138s" totalImagesPullingTime="1.475550791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 12:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 12:59:47.93699064 +0000 UTC m=+788.018431715" watchObservedRunningTime="2026-07-06 12:59:47.937771138 +0000 UTC m=+788.019212166" Jul 06 12:59:53.929657 ip-10-0-136-21 kubenswrapper[2559]: I0706 12:59:53.929625 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7c7f5bd644-7dc2h" Jul 06 13:00:35.607157 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.607110 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj"] Jul 06 13:00:35.609917 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.609897 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.612598 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.612576 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j6ntn\"" Jul 06 13:00:35.613532 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.613507 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 13:00:35.613650 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.613594 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 13:00:35.613650 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.613595 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 13:00:35.625845 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.625823 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj"] Jul 06 13:00:35.654634 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.654605 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.654767 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.654660 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nqpm\" (UniqueName: \"kubernetes.io/projected/a35f4d6c-83a8-4518-826b-020434470243-kube-api-access-2nqpm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.654767 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.654690 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.654767 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.654730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.654767 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.654754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.654914 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.654814 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a35f4d6c-83a8-4518-826b-020434470243-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756125 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756084 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756336 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756157 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nqpm\" (UniqueName: \"kubernetes.io/projected/a35f4d6c-83a8-4518-826b-020434470243-kube-api-access-2nqpm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756336 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756336 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756278 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756336 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756302 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756336 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a35f4d6c-83a8-4518-826b-020434470243-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756655 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756632 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756719 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756651 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.756719 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.756668 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.758656 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.758633 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a35f4d6c-83a8-4518-826b-020434470243-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.758827 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.758811 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a35f4d6c-83a8-4518-826b-020434470243-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.764879 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.764859 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nqpm\" (UniqueName: \"kubernetes.io/projected/a35f4d6c-83a8-4518-826b-020434470243-kube-api-access-2nqpm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fvxfj\" (UID: \"a35f4d6c-83a8-4518-826b-020434470243\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:35.921444 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:35.921412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:36.060757 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:36.060729 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj"] Jul 06 13:00:36.063610 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:00:36.063581 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35f4d6c_83a8_4518_826b_020434470243.slice/crio-f7d8a1098a4c989070ff90db02fcc82704ef1193699be7c53b7ac22ce8407a0f WatchSource:0}: Error finding container f7d8a1098a4c989070ff90db02fcc82704ef1193699be7c53b7ac22ce8407a0f: Status 404 returned error can't find the container with id f7d8a1098a4c989070ff90db02fcc82704ef1193699be7c53b7ac22ce8407a0f Jul 06 13:00:36.065053 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:36.065031 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 13:00:36.144637 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:36.144600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" event={"ID":"a35f4d6c-83a8-4518-826b-020434470243","Type":"ContainerStarted","Data":"f7d8a1098a4c989070ff90db02fcc82704ef1193699be7c53b7ac22ce8407a0f"} Jul 06 13:00:43.179098 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:43.179058 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" event={"ID":"a35f4d6c-83a8-4518-826b-020434470243","Type":"ContainerStarted","Data":"8a275a0edc08bca0b697f5122766414dad597823a48e886949ae2e561490bb8a"} Jul 06 13:00:43.199181 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:43.199125 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" podStartSLOduration=2.076196638 podStartE2EDuration="8.199111096s" totalImagesPullingTime="6.122914458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:00:43.196958324 +0000 UTC m=+843.278399351" watchObservedRunningTime="2026-07-06 13:00:43.199111096 +0000 UTC m=+843.280552147" Jul 06 13:00:45.922101 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:45.922053 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:45.922584 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:45.922132 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:45.935002 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:45.934969 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:46.204378 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:46.204298 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fvxfj" Jul 06 13:00:55.810852 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.810816 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb"] Jul 06 13:00:55.816872 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.816849 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.819332 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.819310 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 13:00:55.825545 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.825516 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb"] Jul 06 13:00:55.849821 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.849782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.850006 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.849841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.850006 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.849861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.850006 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.849879 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.850006 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.849933 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.850006 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.849974 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd2qx\" (UniqueName: \"kubernetes.io/projected/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-kube-api-access-sd2qx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.950968 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.950924 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd2qx\" (UniqueName: \"kubernetes.io/projected/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-kube-api-access-sd2qx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.950968 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.950968 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951031 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951054 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951129 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951537 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951513 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951596 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951564 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.951596 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.951580 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.953429 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.953398 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.953732 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.953713 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:55.959995 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:55.959973 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd2qx\" (UniqueName: \"kubernetes.io/projected/b2fb5584-3ac1-4809-84ad-98c4bbd1445a-kube-api-access-sd2qx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fc2kb\" (UID: \"b2fb5584-3ac1-4809-84ad-98c4bbd1445a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:56.128385 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:56.128300 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:00:56.272997 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:56.272974 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb"] Jul 06 13:00:56.274883 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:00:56.274854 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2fb5584_3ac1_4809_84ad_98c4bbd1445a.slice/crio-f3151a9752d53da30eb82ec587cfa65219d74b35b4e4babedda27b60b5675af0 WatchSource:0}: Error finding container f3151a9752d53da30eb82ec587cfa65219d74b35b4e4babedda27b60b5675af0: Status 404 returned error can't find the container with id f3151a9752d53da30eb82ec587cfa65219d74b35b4e4babedda27b60b5675af0 Jul 06 13:00:57.247793 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:57.247759 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" event={"ID":"b2fb5584-3ac1-4809-84ad-98c4bbd1445a","Type":"ContainerStarted","Data":"b2cf876cb23d227db1a9e677eb4c1aac95ab26fa2260f5e43b916ea4de8af99c"} Jul 06 13:00:57.248206 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:57.247810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" event={"ID":"b2fb5584-3ac1-4809-84ad-98c4bbd1445a","Type":"ContainerStarted","Data":"f3151a9752d53da30eb82ec587cfa65219d74b35b4e4babedda27b60b5675af0"} Jul 06 13:00:57.270032 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:00:57.269971 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" podStartSLOduration=1.976255351 podStartE2EDuration="2.269956112s" totalImagesPullingTime="293.700761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:00:57.267832041 +0000 UTC m=+857.349273070" watchObservedRunningTime="2026-07-06 13:00:57.269956112 +0000 UTC m=+857.351397139" Jul 06 13:01:06.128649 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:06.128612 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:01:06.129027 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:06.128699 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:01:06.140749 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:06.140725 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:01:06.301317 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:06.301289 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fc2kb" Jul 06 13:01:40.401189 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:40.401089 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:01:40.401775 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:40.401189 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:01:40.405627 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:40.405608 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:01:40.405739 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:40.405608 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:01:55.880154 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:55.880118 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54f649865d-5k9c8"] Jul 06 13:01:55.883992 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:55.883969 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:55.895978 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:55.895951 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54f649865d-5k9c8"] Jul 06 13:01:55.988983 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:55.988943 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9kxx\" (UniqueName: \"kubernetes.io/projected/053b7ec9-aa72-41c0-a779-afeab3ebdcfe-kube-api-access-j9kxx\") pod \"authorino-54f649865d-5k9c8\" (UID: \"053b7ec9-aa72-41c0-a779-afeab3ebdcfe\") " pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:55.989209 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:55.989006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/053b7ec9-aa72-41c0-a779-afeab3ebdcfe-tls-cert\") pod \"authorino-54f649865d-5k9c8\" (UID: \"053b7ec9-aa72-41c0-a779-afeab3ebdcfe\") " pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:56.089663 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.089628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9kxx\" (UniqueName: \"kubernetes.io/projected/053b7ec9-aa72-41c0-a779-afeab3ebdcfe-kube-api-access-j9kxx\") pod \"authorino-54f649865d-5k9c8\" (UID: \"053b7ec9-aa72-41c0-a779-afeab3ebdcfe\") " pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:56.089839 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.089680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/053b7ec9-aa72-41c0-a779-afeab3ebdcfe-tls-cert\") pod \"authorino-54f649865d-5k9c8\" (UID: \"053b7ec9-aa72-41c0-a779-afeab3ebdcfe\") " pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:56.092030 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.092007 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/053b7ec9-aa72-41c0-a779-afeab3ebdcfe-tls-cert\") pod \"authorino-54f649865d-5k9c8\" (UID: \"053b7ec9-aa72-41c0-a779-afeab3ebdcfe\") " pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:56.099620 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.099591 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9kxx\" (UniqueName: \"kubernetes.io/projected/053b7ec9-aa72-41c0-a779-afeab3ebdcfe-kube-api-access-j9kxx\") pod \"authorino-54f649865d-5k9c8\" (UID: \"053b7ec9-aa72-41c0-a779-afeab3ebdcfe\") " pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:56.192913 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.192877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f649865d-5k9c8" Jul 06 13:01:56.341514 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.341475 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54f649865d-5k9c8"] Jul 06 13:01:56.346818 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:01:56.346792 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod053b7ec9_aa72_41c0_a779_afeab3ebdcfe.slice/crio-88aeee5c5143c2cf778db97dafa9f928142a55616a83d7eb0f10cfe11061c101 WatchSource:0}: Error finding container 88aeee5c5143c2cf778db97dafa9f928142a55616a83d7eb0f10cfe11061c101: Status 404 returned error can't find the container with id 88aeee5c5143c2cf778db97dafa9f928142a55616a83d7eb0f10cfe11061c101 Jul 06 13:01:56.521924 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:56.521832 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f649865d-5k9c8" event={"ID":"053b7ec9-aa72-41c0-a779-afeab3ebdcfe","Type":"ContainerStarted","Data":"88aeee5c5143c2cf778db97dafa9f928142a55616a83d7eb0f10cfe11061c101"} Jul 06 13:01:57.530409 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:57.530355 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f649865d-5k9c8" event={"ID":"053b7ec9-aa72-41c0-a779-afeab3ebdcfe","Type":"ContainerStarted","Data":"009ffab818944b897a6b796d8b015789721ea2bf2f6d77e97dc465d8ffa878f2"} Jul 06 13:01:57.558052 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:57.557989 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54f649865d-5k9c8" podStartSLOduration=2.089561972 podStartE2EDuration="2.557969563s" totalImagesPullingTime="468.407591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:01:57.557351558 +0000 UTC m=+917.638792612" watchObservedRunningTime="2026-07-06 13:01:57.557969563 +0000 UTC m=+917.639410610" Jul 06 13:01:57.642117 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:57.642079 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bcfccb499-dvmz2"] Jul 06 13:01:57.642303 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:57.642281 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" podUID="f436abd7-3602-41ec-b4ed-35877a49a738" containerName="authorino" containerID="cri-o://61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6" gracePeriod=30 Jul 06 13:01:57.905290 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:57.905261 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 13:01:58.009421 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.009388 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz8ck\" (UniqueName: \"kubernetes.io/projected/f436abd7-3602-41ec-b4ed-35877a49a738-kube-api-access-rz8ck\") pod \"f436abd7-3602-41ec-b4ed-35877a49a738\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " Jul 06 13:01:58.009616 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.009528 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f436abd7-3602-41ec-b4ed-35877a49a738-tls-cert\") pod \"f436abd7-3602-41ec-b4ed-35877a49a738\" (UID: \"f436abd7-3602-41ec-b4ed-35877a49a738\") " Jul 06 13:01:58.012201 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.012156 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f436abd7-3602-41ec-b4ed-35877a49a738-kube-api-access-rz8ck" (OuterVolumeSpecName: "kube-api-access-rz8ck") pod "f436abd7-3602-41ec-b4ed-35877a49a738" (UID: "f436abd7-3602-41ec-b4ed-35877a49a738"). InnerVolumeSpecName "kube-api-access-rz8ck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:01:58.020937 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.020908 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f436abd7-3602-41ec-b4ed-35877a49a738-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f436abd7-3602-41ec-b4ed-35877a49a738" (UID: "f436abd7-3602-41ec-b4ed-35877a49a738"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:01:58.110592 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.110558 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz8ck\" (UniqueName: \"kubernetes.io/projected/f436abd7-3602-41ec-b4ed-35877a49a738-kube-api-access-rz8ck\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:01:58.110592 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.110589 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f436abd7-3602-41ec-b4ed-35877a49a738-tls-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:01:58.536507 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.536468 2559 generic.go:356] "Generic (PLEG): container finished" podID="f436abd7-3602-41ec-b4ed-35877a49a738" containerID="61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6" exitCode=0 Jul 06 13:01:58.536925 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.536536 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" Jul 06 13:01:58.536925 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.536553 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" event={"ID":"f436abd7-3602-41ec-b4ed-35877a49a738","Type":"ContainerDied","Data":"61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6"} Jul 06 13:01:58.536925 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.536598 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bcfccb499-dvmz2" event={"ID":"f436abd7-3602-41ec-b4ed-35877a49a738","Type":"ContainerDied","Data":"bc5064e01eb37e4d17e8a587eb8c717739a77f3f17fb53620cad1637eea1f83a"} Jul 06 13:01:58.536925 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.536616 2559 scope.go:117] "RemoveContainer" containerID="61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6" Jul 06 13:01:58.545551 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.545520 2559 scope.go:117] "RemoveContainer" containerID="61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6" Jul 06 13:01:58.545889 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:01:58.545868 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6\": container with ID starting with 61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6 not found: ID does not exist" containerID="61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6" Jul 06 13:01:58.545961 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.545903 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6"} err="failed to get container status \"61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6\": rpc error: code = NotFound desc = could not find container \"61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6\": container with ID starting with 61e1852cfa9a6237a96e3f616dabece36dcf52d085e032a84d1d81a2a0796ed6 not found: ID does not exist" Jul 06 13:01:58.566082 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.566045 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bcfccb499-dvmz2"] Jul 06 13:01:58.571582 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:01:58.571551 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bcfccb499-dvmz2"] Jul 06 13:02:00.416134 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:02:00.416096 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f436abd7-3602-41ec-b4ed-35877a49a738" path="/var/lib/kubelet/pods/f436abd7-3602-41ec-b4ed-35877a49a738/volumes" Jul 06 13:03:18.834529 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:18.834446 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-775867fb56-sxsjp"] Jul 06 13:03:18.835041 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:18.834650 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-775867fb56-sxsjp" podUID="d142d295-5165-4de9-938e-fd526540d6af" containerName="manager" containerID="cri-o://cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff" gracePeriod=30 Jul 06 13:03:19.087768 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.087695 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 13:03:19.163581 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.163552 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert\") pod \"d142d295-5165-4de9-938e-fd526540d6af\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " Jul 06 13:03:19.163781 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.163716 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7s2g\" (UniqueName: \"kubernetes.io/projected/d142d295-5165-4de9-938e-fd526540d6af-kube-api-access-l7s2g\") pod \"d142d295-5165-4de9-938e-fd526540d6af\" (UID: \"d142d295-5165-4de9-938e-fd526540d6af\") " Jul 06 13:03:19.165657 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.165623 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d142d295-5165-4de9-938e-fd526540d6af-kube-api-access-l7s2g" (OuterVolumeSpecName: "kube-api-access-l7s2g") pod "d142d295-5165-4de9-938e-fd526540d6af" (UID: "d142d295-5165-4de9-938e-fd526540d6af"). InnerVolumeSpecName "kube-api-access-l7s2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:03:19.165787 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.165652 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d142d295-5165-4de9-938e-fd526540d6af" (UID: "d142d295-5165-4de9-938e-fd526540d6af"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:03:19.265125 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.265087 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d142d295-5165-4de9-938e-fd526540d6af-webhook-cert\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:03:19.265125 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.265119 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7s2g\" (UniqueName: \"kubernetes.io/projected/d142d295-5165-4de9-938e-fd526540d6af-kube-api-access-l7s2g\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:03:19.930690 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.930654 2559 generic.go:356] "Generic (PLEG): container finished" podID="d142d295-5165-4de9-938e-fd526540d6af" containerID="cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff" exitCode=0 Jul 06 13:03:19.931130 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.930698 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775867fb56-sxsjp" Jul 06 13:03:19.931130 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.930734 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775867fb56-sxsjp" event={"ID":"d142d295-5165-4de9-938e-fd526540d6af","Type":"ContainerDied","Data":"cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff"} Jul 06 13:03:19.931130 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.930787 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775867fb56-sxsjp" event={"ID":"d142d295-5165-4de9-938e-fd526540d6af","Type":"ContainerDied","Data":"ef3d73ba97c69609f0b9ae637720dfa767d79b1ff10641a59040dd2b9cdfd9d8"} Jul 06 13:03:19.931130 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.930811 2559 scope.go:117] "RemoveContainer" containerID="cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff" Jul 06 13:03:19.940349 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.940330 2559 scope.go:117] "RemoveContainer" containerID="cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff" Jul 06 13:03:19.940622 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:03:19.940604 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff\": container with ID starting with cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff not found: ID does not exist" containerID="cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff" Jul 06 13:03:19.940676 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.940631 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff"} err="failed to get container status \"cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff\": rpc error: code = NotFound desc = could not find container \"cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff\": container with ID starting with cec2c07f215ad2af38bab66627b96b8bed403a0a56bad6fc9bdbbccb3e3774ff not found: ID does not exist" Jul 06 13:03:19.953116 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.953089 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-775867fb56-sxsjp"] Jul 06 13:03:19.958312 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:19.958289 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-775867fb56-sxsjp"] Jul 06 13:03:20.417137 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.417057 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d142d295-5165-4de9-938e-fd526540d6af" path="/var/lib/kubelet/pods/d142d295-5165-4de9-938e-fd526540d6af/volumes" Jul 06 13:03:20.743857 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.743826 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-775867fb56-rbgrf"] Jul 06 13:03:20.744215 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.744204 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f436abd7-3602-41ec-b4ed-35877a49a738" containerName="authorino" Jul 06 13:03:20.744268 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.744217 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f436abd7-3602-41ec-b4ed-35877a49a738" containerName="authorino" Jul 06 13:03:20.744268 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.744239 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d142d295-5165-4de9-938e-fd526540d6af" containerName="manager" Jul 06 13:03:20.744268 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.744244 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d142d295-5165-4de9-938e-fd526540d6af" containerName="manager" Jul 06 13:03:20.744363 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.744300 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f436abd7-3602-41ec-b4ed-35877a49a738" containerName="authorino" Jul 06 13:03:20.744363 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.744309 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d142d295-5165-4de9-938e-fd526540d6af" containerName="manager" Jul 06 13:03:20.748607 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.748590 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:20.751849 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.751830 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2tqdb\"" Jul 06 13:03:20.752598 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.752582 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 13:03:20.752687 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.752672 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 13:03:20.757609 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.757586 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775867fb56-rbgrf"] Jul 06 13:03:20.780977 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.780955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e022082-6115-4fb1-a8c0-0a6916dd3371-webhook-cert\") pod \"maas-controller-775867fb56-rbgrf\" (UID: \"1e022082-6115-4fb1-a8c0-0a6916dd3371\") " pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:20.781131 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.780993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p5w5\" (UniqueName: \"kubernetes.io/projected/1e022082-6115-4fb1-a8c0-0a6916dd3371-kube-api-access-6p5w5\") pod \"maas-controller-775867fb56-rbgrf\" (UID: \"1e022082-6115-4fb1-a8c0-0a6916dd3371\") " pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:20.881875 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.881833 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e022082-6115-4fb1-a8c0-0a6916dd3371-webhook-cert\") pod \"maas-controller-775867fb56-rbgrf\" (UID: \"1e022082-6115-4fb1-a8c0-0a6916dd3371\") " pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:20.882064 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.881884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p5w5\" (UniqueName: \"kubernetes.io/projected/1e022082-6115-4fb1-a8c0-0a6916dd3371-kube-api-access-6p5w5\") pod \"maas-controller-775867fb56-rbgrf\" (UID: \"1e022082-6115-4fb1-a8c0-0a6916dd3371\") " pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:20.884346 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.884314 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e022082-6115-4fb1-a8c0-0a6916dd3371-webhook-cert\") pod \"maas-controller-775867fb56-rbgrf\" (UID: \"1e022082-6115-4fb1-a8c0-0a6916dd3371\") " pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:20.891440 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:20.891415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p5w5\" (UniqueName: \"kubernetes.io/projected/1e022082-6115-4fb1-a8c0-0a6916dd3371-kube-api-access-6p5w5\") pod \"maas-controller-775867fb56-rbgrf\" (UID: \"1e022082-6115-4fb1-a8c0-0a6916dd3371\") " pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:21.059564 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:21.059482 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:21.231835 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:21.231795 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-775867fb56-rbgrf"] Jul 06 13:03:21.234968 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:03:21.234942 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e022082_6115_4fb1_a8c0_0a6916dd3371.slice/crio-2b1b6872e1858e585cd9bcabf81e48772e24e9bb8bdd8a2afebf07d4a4a80233 WatchSource:0}: Error finding container 2b1b6872e1858e585cd9bcabf81e48772e24e9bb8bdd8a2afebf07d4a4a80233: Status 404 returned error can't find the container with id 2b1b6872e1858e585cd9bcabf81e48772e24e9bb8bdd8a2afebf07d4a4a80233 Jul 06 13:03:21.943010 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:21.942973 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775867fb56-rbgrf" event={"ID":"1e022082-6115-4fb1-a8c0-0a6916dd3371","Type":"ContainerStarted","Data":"f4d17c2731885b8f53128d1c597da4d0c534a3e6e26efee8c0b3ed3984e2f64c"} Jul 06 13:03:21.943197 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:21.943063 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-775867fb56-rbgrf" event={"ID":"1e022082-6115-4fb1-a8c0-0a6916dd3371","Type":"ContainerStarted","Data":"2b1b6872e1858e585cd9bcabf81e48772e24e9bb8bdd8a2afebf07d4a4a80233"} Jul 06 13:03:21.943197 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:21.943099 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:03:21.963669 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:21.963358 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-775867fb56-rbgrf" podStartSLOduration=1.432570755 podStartE2EDuration="1.963339491s" totalImagesPullingTime="530.768736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:03:21.962004018 +0000 UTC m=+1002.043445048" watchObservedRunningTime="2026-07-06 13:03:21.963339491 +0000 UTC m=+1002.044780519" Jul 06 13:03:32.954304 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:03:32.954276 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-775867fb56-rbgrf" Jul 06 13:06:40.431983 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:06:40.431955 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:06:40.432962 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:06:40.432945 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:06:40.435888 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:06:40.435868 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:06:40.436795 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:06:40.436772 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:11:40.459651 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:11:40.459626 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:11:40.462073 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:11:40.462049 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:11:40.463407 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:11:40.463383 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:11:40.466010 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:11:40.465991 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:15:27.724728 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.724688 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj"] Jul 06 13:15:27.728789 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.728765 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:27.732245 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.732208 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-f2d3593a-serving-cert\"" Jul 06 13:15:27.741138 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.741116 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj"] Jul 06 13:15:27.909870 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.909829 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf638a9-f96b-4fa1-b041-5368f4b65814-openshift-service-ca\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:27.910036 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.909896 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdb98\" (UniqueName: \"kubernetes.io/projected/dcf638a9-f96b-4fa1-b041-5368f4b65814-kube-api-access-qdb98\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:27.910036 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:27.909945 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf638a9-f96b-4fa1-b041-5368f4b65814-maas-api-tls\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.010949 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.010843 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdb98\" (UniqueName: \"kubernetes.io/projected/dcf638a9-f96b-4fa1-b041-5368f4b65814-kube-api-access-qdb98\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.010949 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.010924 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf638a9-f96b-4fa1-b041-5368f4b65814-maas-api-tls\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.011234 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.011000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf638a9-f96b-4fa1-b041-5368f4b65814-openshift-service-ca\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.011668 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.011642 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf638a9-f96b-4fa1-b041-5368f4b65814-openshift-service-ca\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.013501 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.013475 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf638a9-f96b-4fa1-b041-5368f4b65814-maas-api-tls\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.024635 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.024608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdb98\" (UniqueName: \"kubernetes.io/projected/dcf638a9-f96b-4fa1-b041-5368f4b65814-kube-api-access-qdb98\") pod \"maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.039508 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.039482 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:28.183943 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.183913 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj"] Jul 06 13:15:28.185341 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:15:28.185306 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf638a9_f96b_4fa1_b041_5368f4b65814.slice/crio-958ecdf6354412d8caf2e381a54a47a769a2172410af2e674f43e53a7cdcfd5d WatchSource:0}: Error finding container 958ecdf6354412d8caf2e381a54a47a769a2172410af2e674f43e53a7cdcfd5d: Status 404 returned error can't find the container with id 958ecdf6354412d8caf2e381a54a47a769a2172410af2e674f43e53a7cdcfd5d Jul 06 13:15:28.186766 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.186744 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 13:15:28.396714 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:28.396624 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" event={"ID":"dcf638a9-f96b-4fa1-b041-5368f4b65814","Type":"ContainerStarted","Data":"958ecdf6354412d8caf2e381a54a47a769a2172410af2e674f43e53a7cdcfd5d"} Jul 06 13:15:29.403352 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:29.403318 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" event={"ID":"dcf638a9-f96b-4fa1-b041-5368f4b65814","Type":"ContainerStarted","Data":"bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb"} Jul 06 13:15:29.403737 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:29.403442 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:29.451818 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:29.451763 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" podStartSLOduration=1.915493384 podStartE2EDuration="2.45173701s" totalImagesPullingTime="536.243626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:15:29.450407542 +0000 UTC m=+1729.531848569" watchObservedRunningTime="2026-07-06 13:15:29.45173701 +0000 UTC m=+1729.533178037" Jul 06 13:15:32.924495 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:32.924453 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj"] Jul 06 13:15:32.924963 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:32.924688 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" podUID="dcf638a9-f96b-4fa1-b041-5368f4b65814" containerName="maas-api" containerID="cri-o://bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb" gracePeriod=30 Jul 06 13:15:33.171848 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.171821 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:33.258656 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.258573 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdb98\" (UniqueName: \"kubernetes.io/projected/dcf638a9-f96b-4fa1-b041-5368f4b65814-kube-api-access-qdb98\") pod \"dcf638a9-f96b-4fa1-b041-5368f4b65814\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " Jul 06 13:15:33.258803 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.258665 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf638a9-f96b-4fa1-b041-5368f4b65814-openshift-service-ca\") pod \"dcf638a9-f96b-4fa1-b041-5368f4b65814\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " Jul 06 13:15:33.258803 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.258722 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf638a9-f96b-4fa1-b041-5368f4b65814-maas-api-tls\") pod \"dcf638a9-f96b-4fa1-b041-5368f4b65814\" (UID: \"dcf638a9-f96b-4fa1-b041-5368f4b65814\") " Jul 06 13:15:33.259055 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.259028 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcf638a9-f96b-4fa1-b041-5368f4b65814-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcf638a9-f96b-4fa1-b041-5368f4b65814" (UID: "dcf638a9-f96b-4fa1-b041-5368f4b65814"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:15:33.260694 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.260657 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcf638a9-f96b-4fa1-b041-5368f4b65814-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcf638a9-f96b-4fa1-b041-5368f4b65814" (UID: "dcf638a9-f96b-4fa1-b041-5368f4b65814"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:15:33.260795 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.260724 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcf638a9-f96b-4fa1-b041-5368f4b65814-kube-api-access-qdb98" (OuterVolumeSpecName: "kube-api-access-qdb98") pod "dcf638a9-f96b-4fa1-b041-5368f4b65814" (UID: "dcf638a9-f96b-4fa1-b041-5368f4b65814"). InnerVolumeSpecName "kube-api-access-qdb98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:15:33.359630 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.359588 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcf638a9-f96b-4fa1-b041-5368f4b65814-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:15:33.359630 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.359624 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcf638a9-f96b-4fa1-b041-5368f4b65814-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:15:33.359630 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.359636 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdb98\" (UniqueName: \"kubernetes.io/projected/dcf638a9-f96b-4fa1-b041-5368f4b65814-kube-api-access-qdb98\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:15:33.423477 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.423436 2559 generic.go:356] "Generic (PLEG): container finished" podID="dcf638a9-f96b-4fa1-b041-5368f4b65814" containerID="bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb" exitCode=0 Jul 06 13:15:33.423477 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.423476 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" event={"ID":"dcf638a9-f96b-4fa1-b041-5368f4b65814","Type":"ContainerDied","Data":"bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb"} Jul 06 13:15:33.423721 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.423482 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" Jul 06 13:15:33.423721 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.423516 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj" event={"ID":"dcf638a9-f96b-4fa1-b041-5368f4b65814","Type":"ContainerDied","Data":"958ecdf6354412d8caf2e381a54a47a769a2172410af2e674f43e53a7cdcfd5d"} Jul 06 13:15:33.423721 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.423540 2559 scope.go:117] "RemoveContainer" containerID="bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb" Jul 06 13:15:33.432543 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.432523 2559 scope.go:117] "RemoveContainer" containerID="bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb" Jul 06 13:15:33.432786 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:15:33.432766 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb\": container with ID starting with bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb not found: ID does not exist" containerID="bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb" Jul 06 13:15:33.432866 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.432794 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb"} err="failed to get container status \"bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb\": rpc error: code = NotFound desc = could not find container \"bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb\": container with ID starting with bd0b7e269d3924c9f6f200cde0318cf32a983028aa5008c614f34179242060fb not found: ID does not exist" Jul 06 13:15:33.449965 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.449934 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj"] Jul 06 13:15:33.454434 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:33.454413 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-f2d3593a-75f7755f6-mglqj"] Jul 06 13:15:34.415890 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:34.415859 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcf638a9-f96b-4fa1-b041-5368f4b65814" path="/var/lib/kubelet/pods/dcf638a9-f96b-4fa1-b041-5368f4b65814/volumes" Jul 06 13:15:41.018196 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.018148 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz"] Jul 06 13:15:41.018589 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.018530 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcf638a9-f96b-4fa1-b041-5368f4b65814" containerName="maas-api" Jul 06 13:15:41.018589 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.018543 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcf638a9-f96b-4fa1-b041-5368f4b65814" containerName="maas-api" Jul 06 13:15:41.018686 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.018633 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcf638a9-f96b-4fa1-b041-5368f4b65814" containerName="maas-api" Jul 06 13:15:41.022698 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.022679 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.026246 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.026208 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-aedba5e8-serving-cert\"" Jul 06 13:15:41.035348 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.035322 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz"] Jul 06 13:15:41.125899 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.125849 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6c86c2-d48e-4f60-91b4-0c97da39101d-openshift-service-ca\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.126097 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.125959 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.126097 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.126020 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f92br\" (UniqueName: \"kubernetes.io/projected/2e6c86c2-d48e-4f60-91b4-0c97da39101d-kube-api-access-f92br\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.227260 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.227215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f92br\" (UniqueName: \"kubernetes.io/projected/2e6c86c2-d48e-4f60-91b4-0c97da39101d-kube-api-access-f92br\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.227523 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.227316 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6c86c2-d48e-4f60-91b4-0c97da39101d-openshift-service-ca\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.227523 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.227379 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.227523 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:15:41.227465 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-aedba5e8-serving-cert: secret "maas-api-e2e-ait-aedba5e8-serving-cert" not found Jul 06 13:15:41.227644 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:15:41.227543 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls podName:2e6c86c2-d48e-4f60-91b4-0c97da39101d nodeName:}" failed. No retries permitted until 2026-07-06 13:15:41.727523178 +0000 UTC m=+1741.808964185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls") pod "maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" (UID: "2e6c86c2-d48e-4f60-91b4-0c97da39101d") : secret "maas-api-e2e-ait-aedba5e8-serving-cert" not found Jul 06 13:15:41.227940 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.227922 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6c86c2-d48e-4f60-91b4-0c97da39101d-openshift-service-ca\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.238526 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.238501 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f92br\" (UniqueName: \"kubernetes.io/projected/2e6c86c2-d48e-4f60-91b4-0c97da39101d-kube-api-access-f92br\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.732039 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:41.732004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:41.732248 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:15:41.732176 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-aedba5e8-serving-cert: secret "maas-api-e2e-ait-aedba5e8-serving-cert" not found Jul 06 13:15:41.732248 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:15:41.732236 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls podName:2e6c86c2-d48e-4f60-91b4-0c97da39101d nodeName:}" failed. No retries permitted until 2026-07-06 13:15:42.732220926 +0000 UTC m=+1742.813661935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls") pod "maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" (UID: "2e6c86c2-d48e-4f60-91b4-0c97da39101d") : secret "maas-api-e2e-ait-aedba5e8-serving-cert" not found Jul 06 13:15:42.742950 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:42.742909 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:42.745579 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:42.745549 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") pod \"maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:42.834852 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:42.834816 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:42.991017 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:42.990982 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz"] Jul 06 13:15:42.994592 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:15:42.994517 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6c86c2_d48e_4f60_91b4_0c97da39101d.slice/crio-732849b31e0ae4f760df8a3a71a0c06d2ef1b9d6f473e9c2cf9473ccd306e493 WatchSource:0}: Error finding container 732849b31e0ae4f760df8a3a71a0c06d2ef1b9d6f473e9c2cf9473ccd306e493: Status 404 returned error can't find the container with id 732849b31e0ae4f760df8a3a71a0c06d2ef1b9d6f473e9c2cf9473ccd306e493 Jul 06 13:15:43.472835 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:43.472797 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" event={"ID":"2e6c86c2-d48e-4f60-91b4-0c97da39101d","Type":"ContainerStarted","Data":"a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1"} Jul 06 13:15:43.473045 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:43.472847 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" event={"ID":"2e6c86c2-d48e-4f60-91b4-0c97da39101d","Type":"ContainerStarted","Data":"732849b31e0ae4f760df8a3a71a0c06d2ef1b9d6f473e9c2cf9473ccd306e493"} Jul 06 13:15:43.473045 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:43.472891 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:43.494331 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:43.494283 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" podStartSLOduration=3.177964678 podStartE2EDuration="3.49426782s" totalImagesPullingTime="316.303142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:15:43.493683486 +0000 UTC m=+1743.575124555" watchObservedRunningTime="2026-07-06 13:15:43.49426782 +0000 UTC m=+1743.575708880" Jul 06 13:15:46.420700 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.420314 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz"] Jul 06 13:15:46.420700 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.420523 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" podUID="2e6c86c2-d48e-4f60-91b4-0c97da39101d" containerName="maas-api" containerID="cri-o://a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1" gracePeriod=30 Jul 06 13:15:46.677649 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.677590 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:46.778911 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.778876 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6c86c2-d48e-4f60-91b4-0c97da39101d-openshift-service-ca\") pod \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " Jul 06 13:15:46.779107 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.778952 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f92br\" (UniqueName: \"kubernetes.io/projected/2e6c86c2-d48e-4f60-91b4-0c97da39101d-kube-api-access-f92br\") pod \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " Jul 06 13:15:46.779107 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.778994 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") pod \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\" (UID: \"2e6c86c2-d48e-4f60-91b4-0c97da39101d\") " Jul 06 13:15:46.779330 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.779306 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e6c86c2-d48e-4f60-91b4-0c97da39101d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e6c86c2-d48e-4f60-91b4-0c97da39101d" (UID: "2e6c86c2-d48e-4f60-91b4-0c97da39101d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:15:46.781058 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.781033 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e6c86c2-d48e-4f60-91b4-0c97da39101d" (UID: "2e6c86c2-d48e-4f60-91b4-0c97da39101d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:15:46.781058 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.781043 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e6c86c2-d48e-4f60-91b4-0c97da39101d-kube-api-access-f92br" (OuterVolumeSpecName: "kube-api-access-f92br") pod "2e6c86c2-d48e-4f60-91b4-0c97da39101d" (UID: "2e6c86c2-d48e-4f60-91b4-0c97da39101d"). InnerVolumeSpecName "kube-api-access-f92br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:15:46.880464 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.880432 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f92br\" (UniqueName: \"kubernetes.io/projected/2e6c86c2-d48e-4f60-91b4-0c97da39101d-kube-api-access-f92br\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:15:46.880464 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.880458 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e6c86c2-d48e-4f60-91b4-0c97da39101d-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:15:46.880464 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:46.880469 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e6c86c2-d48e-4f60-91b4-0c97da39101d-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:15:47.491256 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.491222 2559 generic.go:356] "Generic (PLEG): container finished" podID="2e6c86c2-d48e-4f60-91b4-0c97da39101d" containerID="a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1" exitCode=0 Jul 06 13:15:47.491683 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.491268 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" Jul 06 13:15:47.491683 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.491279 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" event={"ID":"2e6c86c2-d48e-4f60-91b4-0c97da39101d","Type":"ContainerDied","Data":"a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1"} Jul 06 13:15:47.491683 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.491323 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz" event={"ID":"2e6c86c2-d48e-4f60-91b4-0c97da39101d","Type":"ContainerDied","Data":"732849b31e0ae4f760df8a3a71a0c06d2ef1b9d6f473e9c2cf9473ccd306e493"} Jul 06 13:15:47.491683 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.491344 2559 scope.go:117] "RemoveContainer" containerID="a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1" Jul 06 13:15:47.499873 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.499849 2559 scope.go:117] "RemoveContainer" containerID="a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1" Jul 06 13:15:47.500115 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:15:47.500093 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1\": container with ID starting with a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1 not found: ID does not exist" containerID="a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1" Jul 06 13:15:47.500186 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.500125 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1"} err="failed to get container status \"a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1\": rpc error: code = NotFound desc = could not find container \"a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1\": container with ID starting with a7c8a0ef4e61decc8ef1dff1e6ab7697041ea185fc000b01ea454d8e5c987ae1 not found: ID does not exist" Jul 06 13:15:47.533304 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.533280 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz"] Jul 06 13:15:47.543815 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:47.543792 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-aedba5e8-7b468854fb-xgppz"] Jul 06 13:15:48.415808 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:15:48.415773 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e6c86c2-d48e-4f60-91b4-0c97da39101d" path="/var/lib/kubelet/pods/2e6c86c2-d48e-4f60-91b4-0c97da39101d/volumes" Jul 06 13:16:00.116612 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.116581 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7"] Jul 06 13:16:00.116986 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.116952 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e6c86c2-d48e-4f60-91b4-0c97da39101d" containerName="maas-api" Jul 06 13:16:00.116986 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.116962 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e6c86c2-d48e-4f60-91b4-0c97da39101d" containerName="maas-api" Jul 06 13:16:00.117103 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.117031 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e6c86c2-d48e-4f60-91b4-0c97da39101d" containerName="maas-api" Jul 06 13:16:00.121334 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.121308 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.124547 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.124520 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-3593f9e9-serving-cert\"" Jul 06 13:16:00.146284 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.146248 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7"] Jul 06 13:16:00.196100 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.196071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d415c021-9f86-41c5-b684-f6027e1d695d-openshift-service-ca\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.196266 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.196146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgrdx\" (UniqueName: \"kubernetes.io/projected/d415c021-9f86-41c5-b684-f6027e1d695d-kube-api-access-kgrdx\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.196266 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.196217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.296796 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.296765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.296970 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.296814 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d415c021-9f86-41c5-b684-f6027e1d695d-openshift-service-ca\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.296970 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.296869 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgrdx\" (UniqueName: \"kubernetes.io/projected/d415c021-9f86-41c5-b684-f6027e1d695d-kube-api-access-kgrdx\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.296970 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:16:00.296924 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-3593f9e9-serving-cert: secret "maas-api-e2e-derive-3593f9e9-serving-cert" not found Jul 06 13:16:00.297102 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:16:00.296990 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls podName:d415c021-9f86-41c5-b684-f6027e1d695d nodeName:}" failed. No retries permitted until 2026-07-06 13:16:00.796972366 +0000 UTC m=+1760.878413371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls") pod "maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" (UID: "d415c021-9f86-41c5-b684-f6027e1d695d") : secret "maas-api-e2e-derive-3593f9e9-serving-cert" not found Jul 06 13:16:00.297442 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.297424 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d415c021-9f86-41c5-b684-f6027e1d695d-openshift-service-ca\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.312774 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.312755 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgrdx\" (UniqueName: \"kubernetes.io/projected/d415c021-9f86-41c5-b684-f6027e1d695d-kube-api-access-kgrdx\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.801752 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.801720 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:00.803983 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:00.803960 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls\") pod \"maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:01.031287 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:01.031253 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:01.185382 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:01.185357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7"] Jul 06 13:16:01.187983 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:16:01.187958 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd415c021_9f86_41c5_b684_f6027e1d695d.slice/crio-a6fe4103d24b7eeb7838b86c9cdc7a8e0de14f4126749a4471e9cc1a62775d7b WatchSource:0}: Error finding container a6fe4103d24b7eeb7838b86c9cdc7a8e0de14f4126749a4471e9cc1a62775d7b: Status 404 returned error can't find the container with id a6fe4103d24b7eeb7838b86c9cdc7a8e0de14f4126749a4471e9cc1a62775d7b Jul 06 13:16:01.555681 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:01.555653 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" event={"ID":"d415c021-9f86-41c5-b684-f6027e1d695d","Type":"ContainerStarted","Data":"a6fe4103d24b7eeb7838b86c9cdc7a8e0de14f4126749a4471e9cc1a62775d7b"} Jul 06 13:16:02.562399 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:02.562364 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" event={"ID":"d415c021-9f86-41c5-b684-f6027e1d695d","Type":"ContainerStarted","Data":"276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e"} Jul 06 13:16:02.562775 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:02.562541 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:02.582281 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:02.582237 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" podStartSLOduration=2.228153482 podStartE2EDuration="2.582220865s" totalImagesPullingTime="354.067383ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:16:02.580720698 +0000 UTC m=+1762.662161748" watchObservedRunningTime="2026-07-06 13:16:02.582220865 +0000 UTC m=+1762.663661891" Jul 06 13:16:04.974692 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:04.974655 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7"] Jul 06 13:16:04.975082 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:04.974854 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" podUID="d415c021-9f86-41c5-b684-f6027e1d695d" containerName="maas-api" containerID="cri-o://276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e" gracePeriod=30 Jul 06 13:16:05.237892 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.237835 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:05.347237 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.347208 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d415c021-9f86-41c5-b684-f6027e1d695d-openshift-service-ca\") pod \"d415c021-9f86-41c5-b684-f6027e1d695d\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " Jul 06 13:16:05.347413 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.347247 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls\") pod \"d415c021-9f86-41c5-b684-f6027e1d695d\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " Jul 06 13:16:05.347413 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.347273 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgrdx\" (UniqueName: \"kubernetes.io/projected/d415c021-9f86-41c5-b684-f6027e1d695d-kube-api-access-kgrdx\") pod \"d415c021-9f86-41c5-b684-f6027e1d695d\" (UID: \"d415c021-9f86-41c5-b684-f6027e1d695d\") " Jul 06 13:16:05.347645 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.347624 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d415c021-9f86-41c5-b684-f6027e1d695d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d415c021-9f86-41c5-b684-f6027e1d695d" (UID: "d415c021-9f86-41c5-b684-f6027e1d695d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:16:05.349469 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.349446 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d415c021-9f86-41c5-b684-f6027e1d695d" (UID: "d415c021-9f86-41c5-b684-f6027e1d695d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:16:05.349469 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.349448 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d415c021-9f86-41c5-b684-f6027e1d695d-kube-api-access-kgrdx" (OuterVolumeSpecName: "kube-api-access-kgrdx") pod "d415c021-9f86-41c5-b684-f6027e1d695d" (UID: "d415c021-9f86-41c5-b684-f6027e1d695d"). InnerVolumeSpecName "kube-api-access-kgrdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:16:05.448062 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.448032 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d415c021-9f86-41c5-b684-f6027e1d695d-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:16:05.448062 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.448057 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d415c021-9f86-41c5-b684-f6027e1d695d-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:16:05.448062 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.448067 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgrdx\" (UniqueName: \"kubernetes.io/projected/d415c021-9f86-41c5-b684-f6027e1d695d-kube-api-access-kgrdx\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:16:05.577600 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.577519 2559 generic.go:356] "Generic (PLEG): container finished" podID="d415c021-9f86-41c5-b684-f6027e1d695d" containerID="276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e" exitCode=0 Jul 06 13:16:05.577600 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.577564 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" Jul 06 13:16:05.577763 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.577608 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" event={"ID":"d415c021-9f86-41c5-b684-f6027e1d695d","Type":"ContainerDied","Data":"276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e"} Jul 06 13:16:05.577763 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.577660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7" event={"ID":"d415c021-9f86-41c5-b684-f6027e1d695d","Type":"ContainerDied","Data":"a6fe4103d24b7eeb7838b86c9cdc7a8e0de14f4126749a4471e9cc1a62775d7b"} Jul 06 13:16:05.577763 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.577681 2559 scope.go:117] "RemoveContainer" containerID="276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e" Jul 06 13:16:05.587224 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.587208 2559 scope.go:117] "RemoveContainer" containerID="276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e" Jul 06 13:16:05.587466 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:16:05.587446 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e\": container with ID starting with 276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e not found: ID does not exist" containerID="276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e" Jul 06 13:16:05.587532 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.587474 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e"} err="failed to get container status \"276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e\": rpc error: code = NotFound desc = could not find container \"276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e\": container with ID starting with 276ffa36a198d323fc74955a6a067d1045a800064fa7cc5d5e7a9830288e596e not found: ID does not exist" Jul 06 13:16:05.610507 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.610487 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7"] Jul 06 13:16:05.614372 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:05.614353 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-3593f9e9-b44dfb7fc-m72h7"] Jul 06 13:16:06.416682 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:06.416650 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d415c021-9f86-41c5-b684-f6027e1d695d" path="/var/lib/kubelet/pods/d415c021-9f86-41c5-b684-f6027e1d695d/volumes" Jul 06 13:16:40.489647 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:40.489616 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:16:40.493009 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:40.492982 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:16:40.493703 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:40.493687 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:16:40.497094 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:16:40.497075 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:17:59.942836 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.942803 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl"] Jul 06 13:17:59.943451 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.943238 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d415c021-9f86-41c5-b684-f6027e1d695d" containerName="maas-api" Jul 06 13:17:59.943451 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.943251 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d415c021-9f86-41c5-b684-f6027e1d695d" containerName="maas-api" Jul 06 13:17:59.943451 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.943321 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d415c021-9f86-41c5-b684-f6027e1d695d" containerName="maas-api" Jul 06 13:17:59.946474 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.946451 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:17:59.951090 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.951066 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-0d2a555d-serving-cert\"" Jul 06 13:17:59.980586 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.980548 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl"] Jul 06 13:17:59.994300 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.994275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8920b-a083-42b1-a411-d5d4629109c9-openshift-service-ca\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:17:59.994434 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.994312 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm2hl\" (UniqueName: \"kubernetes.io/projected/cbb8920b-a083-42b1-a411-d5d4629109c9-kube-api-access-dm2hl\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:17:59.994476 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:17:59.994443 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.095113 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.095076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.095315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.095146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8920b-a083-42b1-a411-d5d4629109c9-openshift-service-ca\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.095315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.095208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm2hl\" (UniqueName: \"kubernetes.io/projected/cbb8920b-a083-42b1-a411-d5d4629109c9-kube-api-access-dm2hl\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.095315 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:18:00.095273 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-0d2a555d-serving-cert: secret "maas-api-e2e-mt-0d2a555d-serving-cert" not found Jul 06 13:18:00.095426 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:18:00.095346 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls podName:cbb8920b-a083-42b1-a411-d5d4629109c9 nodeName:}" failed. No retries permitted until 2026-07-06 13:18:00.595323605 +0000 UTC m=+1880.676764630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls") pod "maas-api-e2e-mt-0d2a555d-5676765896-x66cl" (UID: "cbb8920b-a083-42b1-a411-d5d4629109c9") : secret "maas-api-e2e-mt-0d2a555d-serving-cert" not found Jul 06 13:18:00.095852 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.095825 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8920b-a083-42b1-a411-d5d4629109c9-openshift-service-ca\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.116563 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.116532 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm2hl\" (UniqueName: \"kubernetes.io/projected/cbb8920b-a083-42b1-a411-d5d4629109c9-kube-api-access-dm2hl\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.598706 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.598666 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.601436 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.601412 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls\") pod \"maas-api-e2e-mt-0d2a555d-5676765896-x66cl\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:00.856901 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:00.856810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:01.018602 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:01.018564 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl"] Jul 06 13:18:01.023593 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:18:01.023564 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb8920b_a083_42b1_a411_d5d4629109c9.slice/crio-a7642f6db4e0d60b54d383e482dd0d98cd5bd44ac12b0afe07a53e7efd17fd87 WatchSource:0}: Error finding container a7642f6db4e0d60b54d383e482dd0d98cd5bd44ac12b0afe07a53e7efd17fd87: Status 404 returned error can't find the container with id a7642f6db4e0d60b54d383e482dd0d98cd5bd44ac12b0afe07a53e7efd17fd87 Jul 06 13:18:01.121588 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:01.121505 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" event={"ID":"cbb8920b-a083-42b1-a411-d5d4629109c9","Type":"ContainerStarted","Data":"a7642f6db4e0d60b54d383e482dd0d98cd5bd44ac12b0afe07a53e7efd17fd87"} Jul 06 13:18:02.128182 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:02.128145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" event={"ID":"cbb8920b-a083-42b1-a411-d5d4629109c9","Type":"ContainerStarted","Data":"6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33"} Jul 06 13:18:02.128563 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:02.128235 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:02.189676 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:02.189618 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" podStartSLOduration=2.584479462 podStartE2EDuration="3.189599671s" totalImagesPullingTime="605.120209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:18:02.188095111 +0000 UTC m=+1882.269536158" watchObservedRunningTime="2026-07-06 13:18:02.189599671 +0000 UTC m=+1882.271040698" Jul 06 13:18:08.138149 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:08.138069 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:13.851449 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:13.851412 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl"] Jul 06 13:18:13.851854 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:13.851635 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" podUID="cbb8920b-a083-42b1-a411-d5d4629109c9" containerName="maas-api" containerID="cri-o://6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33" gracePeriod=30 Jul 06 13:18:14.103987 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.103930 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:14.190867 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.190833 2559 generic.go:356] "Generic (PLEG): container finished" podID="cbb8920b-a083-42b1-a411-d5d4629109c9" containerID="6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33" exitCode=0 Jul 06 13:18:14.191037 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.190882 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" Jul 06 13:18:14.191037 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.190932 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" event={"ID":"cbb8920b-a083-42b1-a411-d5d4629109c9","Type":"ContainerDied","Data":"6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33"} Jul 06 13:18:14.191037 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.190994 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl" event={"ID":"cbb8920b-a083-42b1-a411-d5d4629109c9","Type":"ContainerDied","Data":"a7642f6db4e0d60b54d383e482dd0d98cd5bd44ac12b0afe07a53e7efd17fd87"} Jul 06 13:18:14.191037 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.191025 2559 scope.go:117] "RemoveContainer" containerID="6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33" Jul 06 13:18:14.199565 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.199549 2559 scope.go:117] "RemoveContainer" containerID="6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33" Jul 06 13:18:14.199809 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:18:14.199793 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33\": container with ID starting with 6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33 not found: ID does not exist" containerID="6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33" Jul 06 13:18:14.199854 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.199818 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33"} err="failed to get container status \"6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33\": rpc error: code = NotFound desc = could not find container \"6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33\": container with ID starting with 6aec5cee0a8e30694683742e3d2974ccaae7ef3e7cbe6190d9435feba9e1eb33 not found: ID does not exist" Jul 06 13:18:14.228623 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.228586 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm2hl\" (UniqueName: \"kubernetes.io/projected/cbb8920b-a083-42b1-a411-d5d4629109c9-kube-api-access-dm2hl\") pod \"cbb8920b-a083-42b1-a411-d5d4629109c9\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " Jul 06 13:18:14.228853 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.228667 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls\") pod \"cbb8920b-a083-42b1-a411-d5d4629109c9\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " Jul 06 13:18:14.228853 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.228749 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8920b-a083-42b1-a411-d5d4629109c9-openshift-service-ca\") pod \"cbb8920b-a083-42b1-a411-d5d4629109c9\" (UID: \"cbb8920b-a083-42b1-a411-d5d4629109c9\") " Jul 06 13:18:14.229152 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.229122 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb8920b-a083-42b1-a411-d5d4629109c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbb8920b-a083-42b1-a411-d5d4629109c9" (UID: "cbb8920b-a083-42b1-a411-d5d4629109c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:18:14.230745 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.230722 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbb8920b-a083-42b1-a411-d5d4629109c9" (UID: "cbb8920b-a083-42b1-a411-d5d4629109c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:18:14.230827 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.230765 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb8920b-a083-42b1-a411-d5d4629109c9-kube-api-access-dm2hl" (OuterVolumeSpecName: "kube-api-access-dm2hl") pod "cbb8920b-a083-42b1-a411-d5d4629109c9" (UID: "cbb8920b-a083-42b1-a411-d5d4629109c9"). InnerVolumeSpecName "kube-api-access-dm2hl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:18:14.330572 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.330532 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbb8920b-a083-42b1-a411-d5d4629109c9-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:18:14.330572 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.330568 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb8920b-a083-42b1-a411-d5d4629109c9-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:18:14.330572 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.330583 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm2hl\" (UniqueName: \"kubernetes.io/projected/cbb8920b-a083-42b1-a411-d5d4629109c9-kube-api-access-dm2hl\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:18:14.508102 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.508067 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl"] Jul 06 13:18:14.512780 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:14.512744 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-0d2a555d-5676765896-x66cl"] Jul 06 13:18:16.417000 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:16.416960 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb8920b-a083-42b1-a411-d5d4629109c9" path="/var/lib/kubelet/pods/cbb8920b-a083-42b1-a411-d5d4629109c9/volumes" Jul 06 13:18:45.235976 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.235945 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6"] Jul 06 13:18:45.236366 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.236343 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb8920b-a083-42b1-a411-d5d4629109c9" containerName="maas-api" Jul 06 13:18:45.236366 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.236354 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb8920b-a083-42b1-a411-d5d4629109c9" containerName="maas-api" Jul 06 13:18:45.236443 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.236418 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb8920b-a083-42b1-a411-d5d4629109c9" containerName="maas-api" Jul 06 13:18:45.239310 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.239282 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.242351 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.242329 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-69a5e1cc-serving-cert\"" Jul 06 13:18:45.303927 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.303894 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be28cede-93f1-46bf-a207-11c1194ff7a7-openshift-service-ca\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.304081 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.303940 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbvjc\" (UniqueName: \"kubernetes.io/projected/be28cede-93f1-46bf-a207-11c1194ff7a7-kube-api-access-qbvjc\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.304081 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.304029 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be28cede-93f1-46bf-a207-11c1194ff7a7-maas-api-tls\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.361889 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.361863 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6"] Jul 06 13:18:45.405338 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.405306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbvjc\" (UniqueName: \"kubernetes.io/projected/be28cede-93f1-46bf-a207-11c1194ff7a7-kube-api-access-qbvjc\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.405517 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.405386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be28cede-93f1-46bf-a207-11c1194ff7a7-maas-api-tls\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.405517 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.405472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be28cede-93f1-46bf-a207-11c1194ff7a7-openshift-service-ca\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.406101 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.406076 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be28cede-93f1-46bf-a207-11c1194ff7a7-openshift-service-ca\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.407789 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.407769 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be28cede-93f1-46bf-a207-11c1194ff7a7-maas-api-tls\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.417021 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.416999 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbvjc\" (UniqueName: \"kubernetes.io/projected/be28cede-93f1-46bf-a207-11c1194ff7a7-kube-api-access-qbvjc\") pod \"maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.550016 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.549926 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:45.697971 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:45.697946 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6"] Jul 06 13:18:45.698521 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:18:45.698496 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe28cede_93f1_46bf_a207_11c1194ff7a7.slice/crio-42cda1903408be07c8df41e2573c329b7713b6533ee253bbef8a0b8c073449ff WatchSource:0}: Error finding container 42cda1903408be07c8df41e2573c329b7713b6533ee253bbef8a0b8c073449ff: Status 404 returned error can't find the container with id 42cda1903408be07c8df41e2573c329b7713b6533ee253bbef8a0b8c073449ff Jul 06 13:18:46.338958 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:46.338852 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" event={"ID":"be28cede-93f1-46bf-a207-11c1194ff7a7","Type":"ContainerStarted","Data":"72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80"} Jul 06 13:18:46.338958 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:46.338926 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:18:46.338958 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:46.338950 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" event={"ID":"be28cede-93f1-46bf-a207-11c1194ff7a7","Type":"ContainerStarted","Data":"42cda1903408be07c8df41e2573c329b7713b6533ee253bbef8a0b8c073449ff"} Jul 06 13:18:46.366157 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:46.366104 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" podStartSLOduration=0.997563781 podStartE2EDuration="1.366087754s" totalImagesPullingTime="368.523973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:18:46.363933274 +0000 UTC m=+1926.445374322" watchObservedRunningTime="2026-07-06 13:18:46.366087754 +0000 UTC m=+1926.447528780" Jul 06 13:18:52.350651 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:18:52.350615 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:19:22.864424 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:22.864392 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6"] Jul 06 13:19:22.864907 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:22.864610 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" podUID="be28cede-93f1-46bf-a207-11c1194ff7a7" containerName="maas-api" containerID="cri-o://72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80" gracePeriod=30 Jul 06 13:19:23.109141 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.109110 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:19:23.170864 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.170840 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be28cede-93f1-46bf-a207-11c1194ff7a7-openshift-service-ca\") pod \"be28cede-93f1-46bf-a207-11c1194ff7a7\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " Jul 06 13:19:23.171032 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.170881 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be28cede-93f1-46bf-a207-11c1194ff7a7-maas-api-tls\") pod \"be28cede-93f1-46bf-a207-11c1194ff7a7\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " Jul 06 13:19:23.171032 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.170951 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbvjc\" (UniqueName: \"kubernetes.io/projected/be28cede-93f1-46bf-a207-11c1194ff7a7-kube-api-access-qbvjc\") pod \"be28cede-93f1-46bf-a207-11c1194ff7a7\" (UID: \"be28cede-93f1-46bf-a207-11c1194ff7a7\") " Jul 06 13:19:23.171324 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.171297 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be28cede-93f1-46bf-a207-11c1194ff7a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be28cede-93f1-46bf-a207-11c1194ff7a7" (UID: "be28cede-93f1-46bf-a207-11c1194ff7a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:19:23.173085 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.173058 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be28cede-93f1-46bf-a207-11c1194ff7a7-kube-api-access-qbvjc" (OuterVolumeSpecName: "kube-api-access-qbvjc") pod "be28cede-93f1-46bf-a207-11c1194ff7a7" (UID: "be28cede-93f1-46bf-a207-11c1194ff7a7"). InnerVolumeSpecName "kube-api-access-qbvjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:19:23.173085 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.173067 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be28cede-93f1-46bf-a207-11c1194ff7a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be28cede-93f1-46bf-a207-11c1194ff7a7" (UID: "be28cede-93f1-46bf-a207-11c1194ff7a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:19:23.271955 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.271901 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be28cede-93f1-46bf-a207-11c1194ff7a7-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:19:23.271955 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.271949 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be28cede-93f1-46bf-a207-11c1194ff7a7-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:19:23.271955 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.271959 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbvjc\" (UniqueName: \"kubernetes.io/projected/be28cede-93f1-46bf-a207-11c1194ff7a7-kube-api-access-qbvjc\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:19:23.527148 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.527060 2559 generic.go:356] "Generic (PLEG): container finished" podID="be28cede-93f1-46bf-a207-11c1194ff7a7" containerID="72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80" exitCode=0 Jul 06 13:19:23.527148 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.527106 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" Jul 06 13:19:23.527363 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.527148 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" event={"ID":"be28cede-93f1-46bf-a207-11c1194ff7a7","Type":"ContainerDied","Data":"72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80"} Jul 06 13:19:23.527363 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.527212 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6" event={"ID":"be28cede-93f1-46bf-a207-11c1194ff7a7","Type":"ContainerDied","Data":"42cda1903408be07c8df41e2573c329b7713b6533ee253bbef8a0b8c073449ff"} Jul 06 13:19:23.527363 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.527235 2559 scope.go:117] "RemoveContainer" containerID="72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80" Jul 06 13:19:23.536535 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.536520 2559 scope.go:117] "RemoveContainer" containerID="72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80" Jul 06 13:19:23.536806 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:19:23.536787 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80\": container with ID starting with 72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80 not found: ID does not exist" containerID="72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80" Jul 06 13:19:23.536862 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.536814 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80"} err="failed to get container status \"72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80\": rpc error: code = NotFound desc = could not find container \"72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80\": container with ID starting with 72047cf31e6203241f72f0527a2d8c509a6cee8ca190dc6ba97ce871cf59aa80 not found: ID does not exist" Jul 06 13:19:23.550868 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.550815 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6"] Jul 06 13:19:23.553778 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:23.553755 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-69a5e1cc-58656b67-c6fn6"] Jul 06 13:19:24.415952 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:19:24.415916 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be28cede-93f1-46bf-a207-11c1194ff7a7" path="/var/lib/kubelet/pods/be28cede-93f1-46bf-a207-11c1194ff7a7/volumes" Jul 06 13:21:18.926827 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.926788 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l"] Jul 06 13:21:18.927998 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.927973 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be28cede-93f1-46bf-a207-11c1194ff7a7" containerName="maas-api" Jul 06 13:21:18.928109 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.928003 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="be28cede-93f1-46bf-a207-11c1194ff7a7" containerName="maas-api" Jul 06 13:21:18.928262 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.928247 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="be28cede-93f1-46bf-a207-11c1194ff7a7" containerName="maas-api" Jul 06 13:21:18.932514 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.932301 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:18.935501 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.935471 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-8bf2fb-serving-cert\"" Jul 06 13:21:18.944394 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:18.944367 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l"] Jul 06 13:21:19.045744 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.045702 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80dc197e-b4a1-4f27-80be-b66c6b669c27-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.045942 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.045758 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fm28\" (UniqueName: \"kubernetes.io/projected/80dc197e-b4a1-4f27-80be-b66c6b669c27-kube-api-access-6fm28\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.045942 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.045890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80dc197e-b4a1-4f27-80be-b66c6b669c27-maas-api-tls\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.146497 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.146460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80dc197e-b4a1-4f27-80be-b66c6b669c27-maas-api-tls\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.146692 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.146516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80dc197e-b4a1-4f27-80be-b66c6b669c27-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.146692 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.146543 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fm28\" (UniqueName: \"kubernetes.io/projected/80dc197e-b4a1-4f27-80be-b66c6b669c27-kube-api-access-6fm28\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.147251 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.147225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80dc197e-b4a1-4f27-80be-b66c6b669c27-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.148927 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.148905 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80dc197e-b4a1-4f27-80be-b66c6b669c27-maas-api-tls\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.154713 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.154689 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fm28\" (UniqueName: \"kubernetes.io/projected/80dc197e-b4a1-4f27-80be-b66c6b669c27-kube-api-access-6fm28\") pod \"maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.243993 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.243873 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:19.391025 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.390971 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l"] Jul 06 13:21:19.393403 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:21:19.393375 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80dc197e_b4a1_4f27_80be_b66c6b669c27.slice/crio-1a65dcbcf36d5542ea6a3e57968f81eeba4a7ef6fbf68398b9bacbba53c855bb WatchSource:0}: Error finding container 1a65dcbcf36d5542ea6a3e57968f81eeba4a7ef6fbf68398b9bacbba53c855bb: Status 404 returned error can't find the container with id 1a65dcbcf36d5542ea6a3e57968f81eeba4a7ef6fbf68398b9bacbba53c855bb Jul 06 13:21:19.394798 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:19.394775 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 13:21:20.073294 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:20.073258 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" event={"ID":"80dc197e-b4a1-4f27-80be-b66c6b669c27","Type":"ContainerStarted","Data":"8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26"} Jul 06 13:21:20.073716 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:20.073308 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" event={"ID":"80dc197e-b4a1-4f27-80be-b66c6b669c27","Type":"ContainerStarted","Data":"1a65dcbcf36d5542ea6a3e57968f81eeba4a7ef6fbf68398b9bacbba53c855bb"} Jul 06 13:21:20.073716 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:20.073374 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:20.097455 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:20.097384 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" podStartSLOduration=1.642510307 podStartE2EDuration="2.097365899s" totalImagesPullingTime="454.855592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:21:20.094395721 +0000 UTC m=+2080.175836766" watchObservedRunningTime="2026-07-06 13:21:20.097365899 +0000 UTC m=+2080.178806926" Jul 06 13:21:26.084355 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:26.084328 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:21:27.092308 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.092270 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2"] Jul 06 13:21:27.096286 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.096263 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.100507 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.100479 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-cb2d4a-serving-cert\"" Jul 06 13:21:27.111983 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.111957 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2"] Jul 06 13:21:27.225665 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.225622 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrgtt\" (UniqueName: \"kubernetes.io/projected/c0880235-009e-4947-8beb-efbb278383e9-kube-api-access-zrgtt\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.225860 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.225704 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0880235-009e-4947-8beb-efbb278383e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.225860 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.225736 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0880235-009e-4947-8beb-efbb278383e9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.326590 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.326556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0880235-009e-4947-8beb-efbb278383e9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.326796 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.326645 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrgtt\" (UniqueName: \"kubernetes.io/projected/c0880235-009e-4947-8beb-efbb278383e9-kube-api-access-zrgtt\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.326796 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.326707 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0880235-009e-4947-8beb-efbb278383e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.327268 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.327244 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0880235-009e-4947-8beb-efbb278383e9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.329116 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.329092 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0880235-009e-4947-8beb-efbb278383e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.338936 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.338912 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrgtt\" (UniqueName: \"kubernetes.io/projected/c0880235-009e-4947-8beb-efbb278383e9-kube-api-access-zrgtt\") pod \"maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.408092 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.407991 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:27.559863 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:27.559837 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2"] Jul 06 13:21:27.561617 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:21:27.561589 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0880235_009e_4947_8beb_efbb278383e9.slice/crio-522f288cb262fd88f58b524e6d3bcf31b58bf1aa8e56e4f4725f5670b02eb9a5 WatchSource:0}: Error finding container 522f288cb262fd88f58b524e6d3bcf31b58bf1aa8e56e4f4725f5670b02eb9a5: Status 404 returned error can't find the container with id 522f288cb262fd88f58b524e6d3bcf31b58bf1aa8e56e4f4725f5670b02eb9a5 Jul 06 13:21:28.112632 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:28.112539 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" event={"ID":"c0880235-009e-4947-8beb-efbb278383e9","Type":"ContainerStarted","Data":"a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10"} Jul 06 13:21:28.112632 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:28.112597 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" event={"ID":"c0880235-009e-4947-8beb-efbb278383e9","Type":"ContainerStarted","Data":"522f288cb262fd88f58b524e6d3bcf31b58bf1aa8e56e4f4725f5670b02eb9a5"} Jul 06 13:21:28.113119 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:28.112660 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:28.141606 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:28.141552 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" podStartSLOduration=0.841485442 podStartE2EDuration="1.141535046s" totalImagesPullingTime="300.049604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:21:28.140202513 +0000 UTC m=+2088.221643539" watchObservedRunningTime="2026-07-06 13:21:28.141535046 +0000 UTC m=+2088.222976073" Jul 06 13:21:34.122996 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:34.122964 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:21:40.519232 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:40.519201 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:21:40.523147 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:40.523122 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:21:40.523654 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:40.523627 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:21:40.527829 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:21:40.527811 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:22:43.802771 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:43.802670 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l"] Jul 06 13:22:43.803398 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:43.802988 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" podUID="80dc197e-b4a1-4f27-80be-b66c6b669c27" containerName="maas-api" containerID="cri-o://8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26" gracePeriod=30 Jul 06 13:22:44.255524 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.255499 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:22:44.349450 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.349353 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fm28\" (UniqueName: \"kubernetes.io/projected/80dc197e-b4a1-4f27-80be-b66c6b669c27-kube-api-access-6fm28\") pod \"80dc197e-b4a1-4f27-80be-b66c6b669c27\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " Jul 06 13:22:44.349450 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.349427 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80dc197e-b4a1-4f27-80be-b66c6b669c27-maas-api-tls\") pod \"80dc197e-b4a1-4f27-80be-b66c6b669c27\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " Jul 06 13:22:44.349701 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.349490 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80dc197e-b4a1-4f27-80be-b66c6b669c27-openshift-service-ca\") pod \"80dc197e-b4a1-4f27-80be-b66c6b669c27\" (UID: \"80dc197e-b4a1-4f27-80be-b66c6b669c27\") " Jul 06 13:22:44.349856 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.349829 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80dc197e-b4a1-4f27-80be-b66c6b669c27-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80dc197e-b4a1-4f27-80be-b66c6b669c27" (UID: "80dc197e-b4a1-4f27-80be-b66c6b669c27"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:22:44.351596 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.351572 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80dc197e-b4a1-4f27-80be-b66c6b669c27-kube-api-access-6fm28" (OuterVolumeSpecName: "kube-api-access-6fm28") pod "80dc197e-b4a1-4f27-80be-b66c6b669c27" (UID: "80dc197e-b4a1-4f27-80be-b66c6b669c27"). InnerVolumeSpecName "kube-api-access-6fm28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:22:44.351596 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.351584 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80dc197e-b4a1-4f27-80be-b66c6b669c27-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80dc197e-b4a1-4f27-80be-b66c6b669c27" (UID: "80dc197e-b4a1-4f27-80be-b66c6b669c27"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:22:44.451000 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.450967 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fm28\" (UniqueName: \"kubernetes.io/projected/80dc197e-b4a1-4f27-80be-b66c6b669c27-kube-api-access-6fm28\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:22:44.451000 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.450996 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80dc197e-b4a1-4f27-80be-b66c6b669c27-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:22:44.451000 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.451006 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80dc197e-b4a1-4f27-80be-b66c6b669c27-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:22:44.483238 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.483199 2559 generic.go:356] "Generic (PLEG): container finished" podID="80dc197e-b4a1-4f27-80be-b66c6b669c27" containerID="8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26" exitCode=0 Jul 06 13:22:44.483238 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.483244 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" Jul 06 13:22:44.483472 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.483332 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" event={"ID":"80dc197e-b4a1-4f27-80be-b66c6b669c27","Type":"ContainerDied","Data":"8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26"} Jul 06 13:22:44.483472 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.483374 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l" event={"ID":"80dc197e-b4a1-4f27-80be-b66c6b669c27","Type":"ContainerDied","Data":"1a65dcbcf36d5542ea6a3e57968f81eeba4a7ef6fbf68398b9bacbba53c855bb"} Jul 06 13:22:44.483472 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.483397 2559 scope.go:117] "RemoveContainer" containerID="8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26" Jul 06 13:22:44.491939 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.491916 2559 scope.go:117] "RemoveContainer" containerID="8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26" Jul 06 13:22:44.492221 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:22:44.492200 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26\": container with ID starting with 8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26 not found: ID does not exist" containerID="8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26" Jul 06 13:22:44.492287 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.492231 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26"} err="failed to get container status \"8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26\": rpc error: code = NotFound desc = could not find container \"8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26\": container with ID starting with 8208015ac171049d751fe057480f838da0092781135a10171552c8da65824e26 not found: ID does not exist" Jul 06 13:22:44.501146 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.501121 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l"] Jul 06 13:22:44.509596 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:44.509573 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8bf2fb-5644744dd4-xrm2l"] Jul 06 13:22:46.416382 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:22:46.416350 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80dc197e-b4a1-4f27-80be-b66c6b669c27" path="/var/lib/kubelet/pods/80dc197e-b4a1-4f27-80be-b66c6b669c27/volumes" Jul 06 13:23:28.662105 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.662076 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2"] Jul 06 13:23:28.662621 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.662288 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" podUID="c0880235-009e-4947-8beb-efbb278383e9" containerName="maas-api" containerID="cri-o://a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10" gracePeriod=30 Jul 06 13:23:28.913805 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.913748 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:23:28.958346 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.958314 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0880235-009e-4947-8beb-efbb278383e9-maas-api-tls\") pod \"c0880235-009e-4947-8beb-efbb278383e9\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " Jul 06 13:23:28.958511 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.958402 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0880235-009e-4947-8beb-efbb278383e9-openshift-service-ca\") pod \"c0880235-009e-4947-8beb-efbb278383e9\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " Jul 06 13:23:28.958553 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.958539 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrgtt\" (UniqueName: \"kubernetes.io/projected/c0880235-009e-4947-8beb-efbb278383e9-kube-api-access-zrgtt\") pod \"c0880235-009e-4947-8beb-efbb278383e9\" (UID: \"c0880235-009e-4947-8beb-efbb278383e9\") " Jul 06 13:23:28.958797 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.958762 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0880235-009e-4947-8beb-efbb278383e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0880235-009e-4947-8beb-efbb278383e9" (UID: "c0880235-009e-4947-8beb-efbb278383e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:23:28.960578 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.960545 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0880235-009e-4947-8beb-efbb278383e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0880235-009e-4947-8beb-efbb278383e9" (UID: "c0880235-009e-4947-8beb-efbb278383e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:23:28.960578 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:28.960558 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0880235-009e-4947-8beb-efbb278383e9-kube-api-access-zrgtt" (OuterVolumeSpecName: "kube-api-access-zrgtt") pod "c0880235-009e-4947-8beb-efbb278383e9" (UID: "c0880235-009e-4947-8beb-efbb278383e9"). InnerVolumeSpecName "kube-api-access-zrgtt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:23:29.059805 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.059773 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0880235-009e-4947-8beb-efbb278383e9-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:23:29.059805 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.059798 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0880235-009e-4947-8beb-efbb278383e9-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:23:29.059805 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.059808 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrgtt\" (UniqueName: \"kubernetes.io/projected/c0880235-009e-4947-8beb-efbb278383e9-kube-api-access-zrgtt\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:23:29.698228 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.698197 2559 generic.go:356] "Generic (PLEG): container finished" podID="c0880235-009e-4947-8beb-efbb278383e9" containerID="a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10" exitCode=0 Jul 06 13:23:29.698610 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.698296 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" Jul 06 13:23:29.698610 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.698330 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" event={"ID":"c0880235-009e-4947-8beb-efbb278383e9","Type":"ContainerDied","Data":"a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10"} Jul 06 13:23:29.698610 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.698377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2" event={"ID":"c0880235-009e-4947-8beb-efbb278383e9","Type":"ContainerDied","Data":"522f288cb262fd88f58b524e6d3bcf31b58bf1aa8e56e4f4725f5670b02eb9a5"} Jul 06 13:23:29.698610 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.698399 2559 scope.go:117] "RemoveContainer" containerID="a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10" Jul 06 13:23:29.712590 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.712564 2559 scope.go:117] "RemoveContainer" containerID="a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10" Jul 06 13:23:29.712913 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:23:29.712884 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10\": container with ID starting with a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10 not found: ID does not exist" containerID="a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10" Jul 06 13:23:29.713013 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.712916 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10"} err="failed to get container status \"a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10\": rpc error: code = NotFound desc = could not find container \"a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10\": container with ID starting with a089972367b6a940082796d2d00f253f2a83a6fbee7f108386921cba745a1e10 not found: ID does not exist" Jul 06 13:23:29.723798 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.723769 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2"] Jul 06 13:23:29.731029 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:29.731009 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-cb2d4a-6674c96947-qxcg2"] Jul 06 13:23:30.416727 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:30.416690 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0880235-009e-4947-8beb-efbb278383e9" path="/var/lib/kubelet/pods/c0880235-009e-4947-8beb-efbb278383e9/volumes" Jul 06 13:23:39.923387 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923357 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs"] Jul 06 13:23:39.923868 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923851 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80dc197e-b4a1-4f27-80be-b66c6b669c27" containerName="maas-api" Jul 06 13:23:39.923922 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923871 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="80dc197e-b4a1-4f27-80be-b66c6b669c27" containerName="maas-api" Jul 06 13:23:39.923922 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923886 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0880235-009e-4947-8beb-efbb278383e9" containerName="maas-api" Jul 06 13:23:39.923922 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923893 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0880235-009e-4947-8beb-efbb278383e9" containerName="maas-api" Jul 06 13:23:39.924025 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923964 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="80dc197e-b4a1-4f27-80be-b66c6b669c27" containerName="maas-api" Jul 06 13:23:39.924025 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.923971 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0880235-009e-4947-8beb-efbb278383e9" containerName="maas-api" Jul 06 13:23:39.928519 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.928501 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:39.932010 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.931979 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-fdcd8d-serving-cert\"" Jul 06 13:23:39.939352 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:39.939321 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs"] Jul 06 13:23:40.064324 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.064291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.064517 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.064331 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwhw\" (UniqueName: \"kubernetes.io/projected/2d16b979-1e99-475d-bc30-f5ae514424f9-kube-api-access-xmwhw\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.064517 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.064441 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d16b979-1e99-475d-bc30-f5ae514424f9-openshift-service-ca\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.165376 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.165338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d16b979-1e99-475d-bc30-f5ae514424f9-openshift-service-ca\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.165575 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.165395 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.165575 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.165417 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwhw\" (UniqueName: \"kubernetes.io/projected/2d16b979-1e99-475d-bc30-f5ae514424f9-kube-api-access-xmwhw\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.165718 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:23:40.165586 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-fdcd8d-serving-cert: secret "maas-api-e2e-api-a-fdcd8d-serving-cert" not found Jul 06 13:23:40.165718 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:23:40.165663 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls podName:2d16b979-1e99-475d-bc30-f5ae514424f9 nodeName:}" failed. No retries permitted until 2026-07-06 13:23:40.665640237 +0000 UTC m=+2220.747081250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls") pod "maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" (UID: "2d16b979-1e99-475d-bc30-f5ae514424f9") : secret "maas-api-e2e-api-a-fdcd8d-serving-cert" not found Jul 06 13:23:40.165987 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.165964 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d16b979-1e99-475d-bc30-f5ae514424f9-openshift-service-ca\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.181107 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.181038 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwhw\" (UniqueName: \"kubernetes.io/projected/2d16b979-1e99-475d-bc30-f5ae514424f9-kube-api-access-xmwhw\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.670510 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.670472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.672965 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.672937 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls\") pod \"maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.838858 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.838824 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:40.998754 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:40.998721 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs"] Jul 06 13:23:41.002475 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:23:41.002447 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d16b979_1e99_475d_bc30_f5ae514424f9.slice/crio-6445bea020bd1035fd17e7e4ddbbacc6826a25c3b234eb4368ebb127bb20f901 WatchSource:0}: Error finding container 6445bea020bd1035fd17e7e4ddbbacc6826a25c3b234eb4368ebb127bb20f901: Status 404 returned error can't find the container with id 6445bea020bd1035fd17e7e4ddbbacc6826a25c3b234eb4368ebb127bb20f901 Jul 06 13:23:41.760018 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:41.759929 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" event={"ID":"2d16b979-1e99-475d-bc30-f5ae514424f9","Type":"ContainerStarted","Data":"c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae"} Jul 06 13:23:41.760018 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:41.759979 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" event={"ID":"2d16b979-1e99-475d-bc30-f5ae514424f9","Type":"ContainerStarted","Data":"6445bea020bd1035fd17e7e4ddbbacc6826a25c3b234eb4368ebb127bb20f901"} Jul 06 13:23:41.760239 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:41.760021 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:41.784394 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:41.784338 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" podStartSLOduration=2.332751697 podStartE2EDuration="2.78432079s" totalImagesPullingTime="451.569093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:23:41.783630766 +0000 UTC m=+2221.865071815" watchObservedRunningTime="2026-07-06 13:23:41.78432079 +0000 UTC m=+2221.865761817" Jul 06 13:23:47.769667 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:47.769638 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:23:51.883685 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.883653 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c"] Jul 06 13:23:51.887404 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.887380 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:51.890510 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.890487 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-544a3c-serving-cert\"" Jul 06 13:23:51.909686 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.909658 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c"] Jul 06 13:23:51.978246 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.978213 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nd4m\" (UniqueName: \"kubernetes.io/projected/eb5aaf30-6263-466d-bf32-ffb31d736da0-kube-api-access-2nd4m\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:51.978246 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.978251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:51.978516 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:51.978301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5aaf30-6263-466d-bf32-ffb31d736da0-openshift-service-ca\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.079370 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.079327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nd4m\" (UniqueName: \"kubernetes.io/projected/eb5aaf30-6263-466d-bf32-ffb31d736da0-kube-api-access-2nd4m\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.079370 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.079371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.079590 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.079418 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5aaf30-6263-466d-bf32-ffb31d736da0-openshift-service-ca\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.079590 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:23:52.079564 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-544a3c-serving-cert: secret "maas-api-e2e-api-b-544a3c-serving-cert" not found Jul 06 13:23:52.079682 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:23:52.079670 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls podName:eb5aaf30-6263-466d-bf32-ffb31d736da0 nodeName:}" failed. No retries permitted until 2026-07-06 13:23:52.579648263 +0000 UTC m=+2232.661089284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls") pod "maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" (UID: "eb5aaf30-6263-466d-bf32-ffb31d736da0") : secret "maas-api-e2e-api-b-544a3c-serving-cert" not found Jul 06 13:23:52.080122 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.080106 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5aaf30-6263-466d-bf32-ffb31d736da0-openshift-service-ca\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.091024 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.090990 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nd4m\" (UniqueName: \"kubernetes.io/projected/eb5aaf30-6263-466d-bf32-ffb31d736da0-kube-api-access-2nd4m\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.584723 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.584682 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.587210 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.587182 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls\") pod \"maas-api-e2e-api-b-544a3c-68d64bf568-pk66c\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.801035 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.801003 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:52.952570 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:52.952545 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c"] Jul 06 13:23:52.955035 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:23:52.955001 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5aaf30_6263_466d_bf32_ffb31d736da0.slice/crio-869555c94d8c4dfd1cd561e726b36aa1505d47293cfaa53466f470e142a32289 WatchSource:0}: Error finding container 869555c94d8c4dfd1cd561e726b36aa1505d47293cfaa53466f470e142a32289: Status 404 returned error can't find the container with id 869555c94d8c4dfd1cd561e726b36aa1505d47293cfaa53466f470e142a32289 Jul 06 13:23:53.818970 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:53.818936 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" event={"ID":"eb5aaf30-6263-466d-bf32-ffb31d736da0","Type":"ContainerStarted","Data":"80afe644fce6d41bdec782fa61abb0b83cc5071e2ca0ecb82da1556b92950080"} Jul 06 13:23:53.819182 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:53.818993 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:23:53.819182 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:53.819018 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" event={"ID":"eb5aaf30-6263-466d-bf32-ffb31d736da0","Type":"ContainerStarted","Data":"869555c94d8c4dfd1cd561e726b36aa1505d47293cfaa53466f470e142a32289"} Jul 06 13:23:53.860372 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:53.860323 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" podStartSLOduration=2.450051695 podStartE2EDuration="2.860307131s" totalImagesPullingTime="410.255436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:23:53.85906116 +0000 UTC m=+2233.940502188" watchObservedRunningTime="2026-07-06 13:23:53.860307131 +0000 UTC m=+2233.941748159" Jul 06 13:23:59.829414 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:23:59.829383 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:24:05.121270 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.121199 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs"] Jul 06 13:24:05.121632 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.121426 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" podUID="2d16b979-1e99-475d-bc30-f5ae514424f9" containerName="maas-api" containerID="cri-o://c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae" gracePeriod=30 Jul 06 13:24:05.368373 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.368352 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:24:05.404504 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.404430 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d16b979-1e99-475d-bc30-f5ae514424f9-openshift-service-ca\") pod \"2d16b979-1e99-475d-bc30-f5ae514424f9\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " Jul 06 13:24:05.404504 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.404469 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls\") pod \"2d16b979-1e99-475d-bc30-f5ae514424f9\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " Jul 06 13:24:05.404709 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.404519 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmwhw\" (UniqueName: \"kubernetes.io/projected/2d16b979-1e99-475d-bc30-f5ae514424f9-kube-api-access-xmwhw\") pod \"2d16b979-1e99-475d-bc30-f5ae514424f9\" (UID: \"2d16b979-1e99-475d-bc30-f5ae514424f9\") " Jul 06 13:24:05.404789 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.404766 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d16b979-1e99-475d-bc30-f5ae514424f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d16b979-1e99-475d-bc30-f5ae514424f9" (UID: "2d16b979-1e99-475d-bc30-f5ae514424f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:24:05.406593 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.406569 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d16b979-1e99-475d-bc30-f5ae514424f9-kube-api-access-xmwhw" (OuterVolumeSpecName: "kube-api-access-xmwhw") pod "2d16b979-1e99-475d-bc30-f5ae514424f9" (UID: "2d16b979-1e99-475d-bc30-f5ae514424f9"). InnerVolumeSpecName "kube-api-access-xmwhw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:24:05.406733 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.406598 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d16b979-1e99-475d-bc30-f5ae514424f9" (UID: "2d16b979-1e99-475d-bc30-f5ae514424f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:24:05.506108 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.506073 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d16b979-1e99-475d-bc30-f5ae514424f9-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:24:05.506108 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.506103 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmwhw\" (UniqueName: \"kubernetes.io/projected/2d16b979-1e99-475d-bc30-f5ae514424f9-kube-api-access-xmwhw\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:24:05.506108 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.506113 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d16b979-1e99-475d-bc30-f5ae514424f9-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:24:05.876338 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.876304 2559 generic.go:356] "Generic (PLEG): container finished" podID="2d16b979-1e99-475d-bc30-f5ae514424f9" containerID="c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae" exitCode=0 Jul 06 13:24:05.876504 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.876348 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" Jul 06 13:24:05.876504 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.876377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" event={"ID":"2d16b979-1e99-475d-bc30-f5ae514424f9","Type":"ContainerDied","Data":"c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae"} Jul 06 13:24:05.876504 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.876422 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs" event={"ID":"2d16b979-1e99-475d-bc30-f5ae514424f9","Type":"ContainerDied","Data":"6445bea020bd1035fd17e7e4ddbbacc6826a25c3b234eb4368ebb127bb20f901"} Jul 06 13:24:05.876504 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.876447 2559 scope.go:117] "RemoveContainer" containerID="c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae" Jul 06 13:24:05.885271 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.885256 2559 scope.go:117] "RemoveContainer" containerID="c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae" Jul 06 13:24:05.885490 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:24:05.885472 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae\": container with ID starting with c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae not found: ID does not exist" containerID="c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae" Jul 06 13:24:05.885563 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.885503 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae"} err="failed to get container status \"c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae\": rpc error: code = NotFound desc = could not find container \"c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae\": container with ID starting with c4030ee59d2e2cf12fb51d5bf8441889d93a2d1cf56453e734c6383b85ff83ae not found: ID does not exist" Jul 06 13:24:05.922829 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.922802 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs"] Jul 06 13:24:05.926160 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:05.926137 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-fdcd8d-65c648d8f8-gxgfs"] Jul 06 13:24:06.415796 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:06.415765 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d16b979-1e99-475d-bc30-f5ae514424f9" path="/var/lib/kubelet/pods/2d16b979-1e99-475d-bc30-f5ae514424f9/volumes" Jul 06 13:24:13.680479 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.680446 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c"] Jul 06 13:24:13.680836 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.680689 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" podUID="eb5aaf30-6263-466d-bf32-ffb31d736da0" containerName="maas-api" containerID="cri-o://80afe644fce6d41bdec782fa61abb0b83cc5071e2ca0ecb82da1556b92950080" gracePeriod=30 Jul 06 13:24:13.917299 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.917270 2559 generic.go:356] "Generic (PLEG): container finished" podID="eb5aaf30-6263-466d-bf32-ffb31d736da0" containerID="80afe644fce6d41bdec782fa61abb0b83cc5071e2ca0ecb82da1556b92950080" exitCode=0 Jul 06 13:24:13.917422 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.917337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" event={"ID":"eb5aaf30-6263-466d-bf32-ffb31d736da0","Type":"ContainerDied","Data":"80afe644fce6d41bdec782fa61abb0b83cc5071e2ca0ecb82da1556b92950080"} Jul 06 13:24:13.917422 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.917386 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" event={"ID":"eb5aaf30-6263-466d-bf32-ffb31d736da0","Type":"ContainerDied","Data":"869555c94d8c4dfd1cd561e726b36aa1505d47293cfaa53466f470e142a32289"} Jul 06 13:24:13.917422 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.917405 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="869555c94d8c4dfd1cd561e726b36aa1505d47293cfaa53466f470e142a32289" Jul 06 13:24:13.924181 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.924156 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:24:13.981694 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.981630 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5aaf30-6263-466d-bf32-ffb31d736da0-openshift-service-ca\") pod \"eb5aaf30-6263-466d-bf32-ffb31d736da0\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " Jul 06 13:24:13.981873 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.981784 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls\") pod \"eb5aaf30-6263-466d-bf32-ffb31d736da0\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " Jul 06 13:24:13.981873 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.981830 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nd4m\" (UniqueName: \"kubernetes.io/projected/eb5aaf30-6263-466d-bf32-ffb31d736da0-kube-api-access-2nd4m\") pod \"eb5aaf30-6263-466d-bf32-ffb31d736da0\" (UID: \"eb5aaf30-6263-466d-bf32-ffb31d736da0\") " Jul 06 13:24:13.982022 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.981995 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb5aaf30-6263-466d-bf32-ffb31d736da0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb5aaf30-6263-466d-bf32-ffb31d736da0" (UID: "eb5aaf30-6263-466d-bf32-ffb31d736da0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:24:13.982156 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.982130 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5aaf30-6263-466d-bf32-ffb31d736da0-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:24:13.983847 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.983822 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb5aaf30-6263-466d-bf32-ffb31d736da0" (UID: "eb5aaf30-6263-466d-bf32-ffb31d736da0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:24:13.983927 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:13.983844 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5aaf30-6263-466d-bf32-ffb31d736da0-kube-api-access-2nd4m" (OuterVolumeSpecName: "kube-api-access-2nd4m") pod "eb5aaf30-6263-466d-bf32-ffb31d736da0" (UID: "eb5aaf30-6263-466d-bf32-ffb31d736da0"). InnerVolumeSpecName "kube-api-access-2nd4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:24:14.082771 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:14.082749 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5aaf30-6263-466d-bf32-ffb31d736da0-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:24:14.082771 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:14.082769 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nd4m\" (UniqueName: \"kubernetes.io/projected/eb5aaf30-6263-466d-bf32-ffb31d736da0-kube-api-access-2nd4m\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:24:14.921667 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:14.921641 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c" Jul 06 13:24:14.947259 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:14.947229 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c"] Jul 06 13:24:14.953923 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:14.953894 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-544a3c-68d64bf568-pk66c"] Jul 06 13:24:16.416288 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:16.416253 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb5aaf30-6263-466d-bf32-ffb31d736da0" path="/var/lib/kubelet/pods/eb5aaf30-6263-466d-bf32-ffb31d736da0/volumes" Jul 06 13:24:25.138501 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138472 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq"] Jul 06 13:24:25.138899 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138886 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d16b979-1e99-475d-bc30-f5ae514424f9" containerName="maas-api" Jul 06 13:24:25.138943 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138901 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d16b979-1e99-475d-bc30-f5ae514424f9" containerName="maas-api" Jul 06 13:24:25.138943 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138912 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5aaf30-6263-466d-bf32-ffb31d736da0" containerName="maas-api" Jul 06 13:24:25.138943 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138918 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5aaf30-6263-466d-bf32-ffb31d736da0" containerName="maas-api" Jul 06 13:24:25.139031 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138972 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb5aaf30-6263-466d-bf32-ffb31d736da0" containerName="maas-api" Jul 06 13:24:25.139031 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.138979 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d16b979-1e99-475d-bc30-f5ae514424f9" containerName="maas-api" Jul 06 13:24:25.142142 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.142127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.144769 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.144748 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-164b27-serving-cert\"" Jul 06 13:24:25.157329 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.157305 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq"] Jul 06 13:24:25.286852 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.286822 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bc6a94-fa2a-4110-9459-fea43db5a4b8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.287058 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.286867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgmmx\" (UniqueName: \"kubernetes.io/projected/70bc6a94-fa2a-4110-9459-fea43db5a4b8-kube-api-access-pgmmx\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.287058 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.286931 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70bc6a94-fa2a-4110-9459-fea43db5a4b8-maas-api-tls\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.388407 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.388362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmmx\" (UniqueName: \"kubernetes.io/projected/70bc6a94-fa2a-4110-9459-fea43db5a4b8-kube-api-access-pgmmx\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.388407 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.388414 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70bc6a94-fa2a-4110-9459-fea43db5a4b8-maas-api-tls\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.388712 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.388520 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bc6a94-fa2a-4110-9459-fea43db5a4b8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.389238 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.389211 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bc6a94-fa2a-4110-9459-fea43db5a4b8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.390988 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.390970 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70bc6a94-fa2a-4110-9459-fea43db5a4b8-maas-api-tls\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.400743 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.400715 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgmmx\" (UniqueName: \"kubernetes.io/projected/70bc6a94-fa2a-4110-9459-fea43db5a4b8-kube-api-access-pgmmx\") pod \"maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.452145 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.452112 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:25.612781 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.612760 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq"] Jul 06 13:24:25.614772 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:24:25.614733 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70bc6a94_fa2a_4110_9459_fea43db5a4b8.slice/crio-9b4c1430779baa9e9f5606731eee3b8b418ae2d8c0b5386c47d4713c04dc0de9 WatchSource:0}: Error finding container 9b4c1430779baa9e9f5606731eee3b8b418ae2d8c0b5386c47d4713c04dc0de9: Status 404 returned error can't find the container with id 9b4c1430779baa9e9f5606731eee3b8b418ae2d8c0b5386c47d4713c04dc0de9 Jul 06 13:24:25.972611 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:25.972587 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" event={"ID":"70bc6a94-fa2a-4110-9459-fea43db5a4b8","Type":"ContainerStarted","Data":"9b4c1430779baa9e9f5606731eee3b8b418ae2d8c0b5386c47d4713c04dc0de9"} Jul 06 13:24:26.978045 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:26.978007 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" event={"ID":"70bc6a94-fa2a-4110-9459-fea43db5a4b8","Type":"ContainerStarted","Data":"a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff"} Jul 06 13:24:26.978440 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:26.978131 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:27.011302 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:27.011253 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" podStartSLOduration=1.6773325209999999 podStartE2EDuration="2.011237847s" totalImagesPullingTime="333.905326ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:24:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:24:27.010292854 +0000 UTC m=+2267.091733884" watchObservedRunningTime="2026-07-06 13:24:27.011237847 +0000 UTC m=+2267.092678874" Jul 06 13:24:32.988840 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:32.988810 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:24:34.668618 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.668571 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm"] Jul 06 13:24:34.672396 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.672374 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.680808 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.680580 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-03b528-serving-cert\"" Jul 06 13:24:34.694003 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.693978 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm"] Jul 06 13:24:34.772151 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.772111 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef921e7f-2701-4d56-86e5-e1aa90af75ab-openshift-service-ca\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.772151 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.772156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.772387 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.772279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk968\" (UniqueName: \"kubernetes.io/projected/ef921e7f-2701-4d56-86e5-e1aa90af75ab-kube-api-access-pk968\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.873446 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.873409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef921e7f-2701-4d56-86e5-e1aa90af75ab-openshift-service-ca\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.873446 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.873452 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.873687 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.873507 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk968\" (UniqueName: \"kubernetes.io/projected/ef921e7f-2701-4d56-86e5-e1aa90af75ab-kube-api-access-pk968\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.873687 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:24:34.873626 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-03b528-serving-cert: secret "maas-api-e2e-shared-b-03b528-serving-cert" not found Jul 06 13:24:34.873778 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:24:34.873701 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls podName:ef921e7f-2701-4d56-86e5-e1aa90af75ab nodeName:}" failed. No retries permitted until 2026-07-06 13:24:35.373683187 +0000 UTC m=+2275.455124193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls") pod "maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" (UID: "ef921e7f-2701-4d56-86e5-e1aa90af75ab") : secret "maas-api-e2e-shared-b-03b528-serving-cert" not found Jul 06 13:24:34.874111 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.874093 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef921e7f-2701-4d56-86e5-e1aa90af75ab-openshift-service-ca\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:34.890183 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:34.890136 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk968\" (UniqueName: \"kubernetes.io/projected/ef921e7f-2701-4d56-86e5-e1aa90af75ab-kube-api-access-pk968\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:35.378184 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:35.378121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:35.380750 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:35.380719 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls\") pod \"maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:35.584521 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:35.584489 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:35.739300 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:35.739273 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm"] Jul 06 13:24:35.742539 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:24:35.742508 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef921e7f_2701_4d56_86e5_e1aa90af75ab.slice/crio-76ceaa36e3dd542d1f1bf073918587dd950aa249e66c0f7aa5a75e65ad537fe9 WatchSource:0}: Error finding container 76ceaa36e3dd542d1f1bf073918587dd950aa249e66c0f7aa5a75e65ad537fe9: Status 404 returned error can't find the container with id 76ceaa36e3dd542d1f1bf073918587dd950aa249e66c0f7aa5a75e65ad537fe9 Jul 06 13:24:36.022237 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:36.022204 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" event={"ID":"ef921e7f-2701-4d56-86e5-e1aa90af75ab","Type":"ContainerStarted","Data":"76ceaa36e3dd542d1f1bf073918587dd950aa249e66c0f7aa5a75e65ad537fe9"} Jul 06 13:24:37.028993 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:37.028954 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" event={"ID":"ef921e7f-2701-4d56-86e5-e1aa90af75ab","Type":"ContainerStarted","Data":"cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f"} Jul 06 13:24:37.029397 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:37.029057 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:43.039434 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:43.039404 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:24:43.061157 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:24:43.061093 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" podStartSLOduration=8.740262896 podStartE2EDuration="9.061077373s" totalImagesPullingTime="320.814477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:24:37.05336701 +0000 UTC m=+2277.134808038" watchObservedRunningTime="2026-07-06 13:24:43.061077373 +0000 UTC m=+2283.142518399" Jul 06 13:26:40.551838 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:26:40.551811 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:26:40.555660 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:26:40.555637 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:26:40.557998 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:26:40.557976 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:26:40.561843 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:26:40.561827 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tgmrj_41e2b5df-fdbb-491d-962e-829c3f6f3405/ovn-acl-logging/0.log" Jul 06 13:29:36.430898 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.430868 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq"] Jul 06 13:29:36.431366 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.431054 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" podUID="70bc6a94-fa2a-4110-9459-fea43db5a4b8" containerName="maas-api" containerID="cri-o://a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff" gracePeriod=30 Jul 06 13:29:36.673795 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.673768 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:29:36.840294 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.840217 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgmmx\" (UniqueName: \"kubernetes.io/projected/70bc6a94-fa2a-4110-9459-fea43db5a4b8-kube-api-access-pgmmx\") pod \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " Jul 06 13:29:36.840294 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.840280 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bc6a94-fa2a-4110-9459-fea43db5a4b8-openshift-service-ca\") pod \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " Jul 06 13:29:36.840484 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.840387 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70bc6a94-fa2a-4110-9459-fea43db5a4b8-maas-api-tls\") pod \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\" (UID: \"70bc6a94-fa2a-4110-9459-fea43db5a4b8\") " Jul 06 13:29:36.840605 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.840574 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70bc6a94-fa2a-4110-9459-fea43db5a4b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70bc6a94-fa2a-4110-9459-fea43db5a4b8" (UID: "70bc6a94-fa2a-4110-9459-fea43db5a4b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:29:36.840749 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.840736 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70bc6a94-fa2a-4110-9459-fea43db5a4b8-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:29:36.842262 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.842241 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70bc6a94-fa2a-4110-9459-fea43db5a4b8-kube-api-access-pgmmx" (OuterVolumeSpecName: "kube-api-access-pgmmx") pod "70bc6a94-fa2a-4110-9459-fea43db5a4b8" (UID: "70bc6a94-fa2a-4110-9459-fea43db5a4b8"). InnerVolumeSpecName "kube-api-access-pgmmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:29:36.842395 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.842358 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70bc6a94-fa2a-4110-9459-fea43db5a4b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70bc6a94-fa2a-4110-9459-fea43db5a4b8" (UID: "70bc6a94-fa2a-4110-9459-fea43db5a4b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:29:36.942123 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.942094 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgmmx\" (UniqueName: \"kubernetes.io/projected/70bc6a94-fa2a-4110-9459-fea43db5a4b8-kube-api-access-pgmmx\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:29:36.942123 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:36.942119 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70bc6a94-fa2a-4110-9459-fea43db5a4b8-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:29:37.487719 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.487686 2559 generic.go:356] "Generic (PLEG): container finished" podID="70bc6a94-fa2a-4110-9459-fea43db5a4b8" containerID="a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff" exitCode=0 Jul 06 13:29:37.488233 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.487729 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" Jul 06 13:29:37.488233 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.487784 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" event={"ID":"70bc6a94-fa2a-4110-9459-fea43db5a4b8","Type":"ContainerDied","Data":"a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff"} Jul 06 13:29:37.488233 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.487847 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq" event={"ID":"70bc6a94-fa2a-4110-9459-fea43db5a4b8","Type":"ContainerDied","Data":"9b4c1430779baa9e9f5606731eee3b8b418ae2d8c0b5386c47d4713c04dc0de9"} Jul 06 13:29:37.488233 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.487884 2559 scope.go:117] "RemoveContainer" containerID="a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff" Jul 06 13:29:37.497444 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.497428 2559 scope.go:117] "RemoveContainer" containerID="a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff" Jul 06 13:29:37.497702 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:29:37.497685 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff\": container with ID starting with a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff not found: ID does not exist" containerID="a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff" Jul 06 13:29:37.497747 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.497710 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff"} err="failed to get container status \"a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff\": rpc error: code = NotFound desc = could not find container \"a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff\": container with ID starting with a19ab69a5869b01c0f9961928c61b59a4fae9656798c1ca3d2f2fc88734c1cff not found: ID does not exist" Jul 06 13:29:37.517069 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.517045 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq"] Jul 06 13:29:37.520382 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:37.520362 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-164b27-5d7f557bfd-764mq"] Jul 06 13:29:38.415808 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:29:38.415777 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70bc6a94-fa2a-4110-9459-fea43db5a4b8" path="/var/lib/kubelet/pods/70bc6a94-fa2a-4110-9459-fea43db5a4b8/volumes" Jul 06 13:30:19.742429 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:19.742344 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm"] Jul 06 13:30:19.742906 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:19.742564 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" podUID="ef921e7f-2701-4d56-86e5-e1aa90af75ab" containerName="maas-api" containerID="cri-o://cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f" gracePeriod=30 Jul 06 13:30:19.991364 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:19.991340 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:30:20.132738 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.132639 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls\") pod \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " Jul 06 13:30:20.132738 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.132694 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk968\" (UniqueName: \"kubernetes.io/projected/ef921e7f-2701-4d56-86e5-e1aa90af75ab-kube-api-access-pk968\") pod \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " Jul 06 13:30:20.132738 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.132731 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef921e7f-2701-4d56-86e5-e1aa90af75ab-openshift-service-ca\") pod \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\" (UID: \"ef921e7f-2701-4d56-86e5-e1aa90af75ab\") " Jul 06 13:30:20.133123 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.133099 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef921e7f-2701-4d56-86e5-e1aa90af75ab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef921e7f-2701-4d56-86e5-e1aa90af75ab" (UID: "ef921e7f-2701-4d56-86e5-e1aa90af75ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 13:30:20.134880 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.134858 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef921e7f-2701-4d56-86e5-e1aa90af75ab" (UID: "ef921e7f-2701-4d56-86e5-e1aa90af75ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 13:30:20.134953 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.134919 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef921e7f-2701-4d56-86e5-e1aa90af75ab-kube-api-access-pk968" (OuterVolumeSpecName: "kube-api-access-pk968") pod "ef921e7f-2701-4d56-86e5-e1aa90af75ab" (UID: "ef921e7f-2701-4d56-86e5-e1aa90af75ab"). InnerVolumeSpecName "kube-api-access-pk968". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 13:30:20.234315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.234284 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef921e7f-2701-4d56-86e5-e1aa90af75ab-openshift-service-ca\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:30:20.234315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.234311 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef921e7f-2701-4d56-86e5-e1aa90af75ab-maas-api-tls\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:30:20.234315 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.234321 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk968\" (UniqueName: \"kubernetes.io/projected/ef921e7f-2701-4d56-86e5-e1aa90af75ab-kube-api-access-pk968\") on node \"ip-10-0-136-21.ec2.internal\" DevicePath \"\"" Jul 06 13:30:20.694433 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.694337 2559 generic.go:356] "Generic (PLEG): container finished" podID="ef921e7f-2701-4d56-86e5-e1aa90af75ab" containerID="cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f" exitCode=0 Jul 06 13:30:20.694433 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.694394 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" Jul 06 13:30:20.694433 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.694411 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" event={"ID":"ef921e7f-2701-4d56-86e5-e1aa90af75ab","Type":"ContainerDied","Data":"cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f"} Jul 06 13:30:20.694718 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.694470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm" event={"ID":"ef921e7f-2701-4d56-86e5-e1aa90af75ab","Type":"ContainerDied","Data":"76ceaa36e3dd542d1f1bf073918587dd950aa249e66c0f7aa5a75e65ad537fe9"} Jul 06 13:30:20.694718 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.694498 2559 scope.go:117] "RemoveContainer" containerID="cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f" Jul 06 13:30:20.705512 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.704043 2559 scope.go:117] "RemoveContainer" containerID="cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f" Jul 06 13:30:20.705718 ip-10-0-136-21 kubenswrapper[2559]: E0706 13:30:20.705692 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f\": container with ID starting with cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f not found: ID does not exist" containerID="cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f" Jul 06 13:30:20.705800 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.705728 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f"} err="failed to get container status \"cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f\": rpc error: code = NotFound desc = could not find container \"cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f\": container with ID starting with cb3dc0dc1f88111b8c405f8bbf9adb784d6a993095fcaa73a2ac3c920b71b19f not found: ID does not exist" Jul 06 13:30:20.714303 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.714271 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm"] Jul 06 13:30:20.720193 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:20.720154 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-03b528-5f9ccd879-xdsgm"] Jul 06 13:30:22.416219 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:22.416185 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef921e7f-2701-4d56-86e5-e1aa90af75ab" path="/var/lib/kubelet/pods/ef921e7f-2701-4d56-86e5-e1aa90af75ab/volumes" Jul 06 13:30:27.483253 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:27.483218 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54f649865d-5k9c8_053b7ec9-aa72-41c0-a779-afeab3ebdcfe/authorino/0.log" Jul 06 13:30:32.682521 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:32.682487 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7c7f5bd644-7dc2h_44400754-fe37-4084-a232-b78c08b77b84/maas-api/0.log" Jul 06 13:30:33.165594 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:33.165511 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-775867fb56-rbgrf_1e022082-6115-4fb1-a8c0-0a6916dd3371/manager/0.log" Jul 06 13:30:33.684601 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:33.684576 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58899bc45d-ts8m8_86603566-35bd-4275-929e-795ca6acdf60/copy-manifests/0.log" Jul 06 13:30:33.693317 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:33.693287 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58899bc45d-ts8m8_86603566-35bd-4275-929e-795ca6acdf60/manager/0.log" Jul 06 13:30:35.336460 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:35.336429 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54f649865d-5k9c8_053b7ec9-aa72-41c0-a779-afeab3ebdcfe/authorino/0.log" Jul 06 13:30:35.475790 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:35.475760 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r9wbj_6a442a44-f29b-4748-bc96-2ff1195dfeee/manager/0.log" Jul 06 13:30:36.906962 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:36.906920 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7965d5bf57-8jf4q_99201dad-aaa5-45d7-831d-f804fb332f3c/kube-auth-proxy/0.log" Jul 06 13:30:38.115926 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:38.115898 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-fvxfj_a35f4d6c-83a8-4518-826b-020434470243/main/0.log" Jul 06 13:30:38.351987 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:38.351958 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-fc2kb_b2fb5584-3ac1-4809-84ad-98c4bbd1445a/main/0.log" Jul 06 13:30:40.718396 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:40.718366 2559 scope.go:117] "RemoveContainer" containerID="80afe644fce6d41bdec782fa61abb0b83cc5071e2ca0ecb82da1556b92950080" Jul 06 13:30:42.771506 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.771475 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-89xhc/must-gather-4jn2w"] Jul 06 13:30:42.771902 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.771889 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70bc6a94-fa2a-4110-9459-fea43db5a4b8" containerName="maas-api" Jul 06 13:30:42.771902 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.771901 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="70bc6a94-fa2a-4110-9459-fea43db5a4b8" containerName="maas-api" Jul 06 13:30:42.771974 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.771918 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef921e7f-2701-4d56-86e5-e1aa90af75ab" containerName="maas-api" Jul 06 13:30:42.771974 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.771923 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef921e7f-2701-4d56-86e5-e1aa90af75ab" containerName="maas-api" Jul 06 13:30:42.772041 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.772003 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="70bc6a94-fa2a-4110-9459-fea43db5a4b8" containerName="maas-api" Jul 06 13:30:42.772041 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.772016 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef921e7f-2701-4d56-86e5-e1aa90af75ab" containerName="maas-api" Jul 06 13:30:42.774735 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.774716 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:42.777797 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.777777 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-89xhc\"/\"kube-root-ca.crt\"" Jul 06 13:30:42.778048 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.778027 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-89xhc\"/\"default-dockercfg-bwj8j\"" Jul 06 13:30:42.778253 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.778236 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-89xhc\"/\"openshift-service-ca.crt\"" Jul 06 13:30:42.790106 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.790086 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-89xhc/must-gather-4jn2w"] Jul 06 13:30:42.837664 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.837638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a05427f-7e72-4c45-a0ae-416d848c8b0f-must-gather-output\") pod \"must-gather-4jn2w\" (UID: \"5a05427f-7e72-4c45-a0ae-416d848c8b0f\") " pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:42.837830 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.837688 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxbt9\" (UniqueName: \"kubernetes.io/projected/5a05427f-7e72-4c45-a0ae-416d848c8b0f-kube-api-access-mxbt9\") pod \"must-gather-4jn2w\" (UID: \"5a05427f-7e72-4c45-a0ae-416d848c8b0f\") " pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:42.939150 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.939116 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a05427f-7e72-4c45-a0ae-416d848c8b0f-must-gather-output\") pod \"must-gather-4jn2w\" (UID: \"5a05427f-7e72-4c45-a0ae-416d848c8b0f\") " pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:42.939331 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.939197 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxbt9\" (UniqueName: \"kubernetes.io/projected/5a05427f-7e72-4c45-a0ae-416d848c8b0f-kube-api-access-mxbt9\") pod \"must-gather-4jn2w\" (UID: \"5a05427f-7e72-4c45-a0ae-416d848c8b0f\") " pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:42.939457 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.939436 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a05427f-7e72-4c45-a0ae-416d848c8b0f-must-gather-output\") pod \"must-gather-4jn2w\" (UID: \"5a05427f-7e72-4c45-a0ae-416d848c8b0f\") " pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:42.951328 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:42.951300 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxbt9\" (UniqueName: \"kubernetes.io/projected/5a05427f-7e72-4c45-a0ae-416d848c8b0f-kube-api-access-mxbt9\") pod \"must-gather-4jn2w\" (UID: \"5a05427f-7e72-4c45-a0ae-416d848c8b0f\") " pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:43.083493 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:43.083433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-89xhc/must-gather-4jn2w" Jul 06 13:30:43.227569 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:43.227548 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-89xhc/must-gather-4jn2w"] Jul 06 13:30:43.229852 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:30:43.229817 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a05427f_7e72_4c45_a0ae_416d848c8b0f.slice/crio-8716f723f222c1b12ba1338bf822009d8820345ce62a6846a334d191ae89b68b WatchSource:0}: Error finding container 8716f723f222c1b12ba1338bf822009d8820345ce62a6846a334d191ae89b68b: Status 404 returned error can't find the container with id 8716f723f222c1b12ba1338bf822009d8820345ce62a6846a334d191ae89b68b Jul 06 13:30:43.231784 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:43.231767 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 13:30:43.803149 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:43.803115 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89xhc/must-gather-4jn2w" event={"ID":"5a05427f-7e72-4c45-a0ae-416d848c8b0f","Type":"ContainerStarted","Data":"8716f723f222c1b12ba1338bf822009d8820345ce62a6846a334d191ae89b68b"} Jul 06 13:30:44.810672 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:44.810633 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89xhc/must-gather-4jn2w" event={"ID":"5a05427f-7e72-4c45-a0ae-416d848c8b0f","Type":"ContainerStarted","Data":"3831e00529f1de8e17300f6524dbdc92e928c897928d22f2101e5e8daf577fe6"} Jul 06 13:30:44.811142 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:44.810689 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89xhc/must-gather-4jn2w" event={"ID":"5a05427f-7e72-4c45-a0ae-416d848c8b0f","Type":"ContainerStarted","Data":"335594b2cc7940ce496fa45087bc51a0e04331588e54648f22a290b8afbfa76d"} Jul 06 13:30:44.847806 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:44.847757 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-89xhc/must-gather-4jn2w" podStartSLOduration=2.201880278 podStartE2EDuration="2.847732018s" totalImagesPullingTime="645.85174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 13:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:30:44.847427025 +0000 UTC m=+2644.928868053" watchObservedRunningTime="2026-07-06 13:30:44.847732018 +0000 UTC m=+2644.929173045" Jul 06 13:30:45.734472 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:45.734440 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jkfvl_956e4c01-2aab-4200-8d08-b03df4427e55/global-pull-secret-syncer/0.log" Jul 06 13:30:45.975715 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:45.975647 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z4j4h_850b8299-9c34-4e86-9d54-95161af94888/konnectivity-agent/0.log" Jul 06 13:30:46.033616 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:46.033538 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-21.ec2.internal_99f360b5a4f3a206c7dffeae18c1b293/haproxy/0.log" Jul 06 13:30:51.083448 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:51.083404 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54f649865d-5k9c8_053b7ec9-aa72-41c0-a779-afeab3ebdcfe/authorino/0.log" Jul 06 13:30:51.136157 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:51.136123 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r9wbj_6a442a44-f29b-4748-bc96-2ff1195dfeee/manager/0.log" Jul 06 13:30:53.387084 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.387055 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/alertmanager/0.log" Jul 06 13:30:53.428867 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.428836 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/config-reloader/0.log" Jul 06 13:30:53.473602 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.473570 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/kube-rbac-proxy-web/0.log" Jul 06 13:30:53.516969 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.516892 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/kube-rbac-proxy/0.log" Jul 06 13:30:53.575362 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.575328 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/kube-rbac-proxy-metric/0.log" Jul 06 13:30:53.615474 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.615448 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/prom-label-proxy/0.log" Jul 06 13:30:53.654435 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.654403 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d2d4aa7-1246-425d-94a1-59352e5ac1b4/init-config-reloader/0.log" Jul 06 13:30:53.718204 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.718160 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-fr49d_73d957ac-833e-4def-a90a-0f720294def7/cluster-monitoring-operator/0.log" Jul 06 13:30:53.927733 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:53.927703 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-649c995458-8xxxk_2d620d2f-94b4-4090-85ef-4ff6c237b25b/metrics-server/0.log" Jul 06 13:30:54.026668 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.026622 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44xk2_9e6d8030-618d-448b-ac5c-050744d195a2/node-exporter/0.log" Jul 06 13:30:54.070052 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.070021 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44xk2_9e6d8030-618d-448b-ac5c-050744d195a2/kube-rbac-proxy/0.log" Jul 06 13:30:54.116491 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.116464 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44xk2_9e6d8030-618d-448b-ac5c-050744d195a2/init-textfile/0.log" Jul 06 13:30:54.385033 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.384939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-wgmgp_693f53d4-7f20-47dd-b4d4-48e57e5eb60f/kube-rbac-proxy-main/0.log" Jul 06 13:30:54.419375 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.419325 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-wgmgp_693f53d4-7f20-47dd-b4d4-48e57e5eb60f/kube-rbac-proxy-self/0.log" Jul 06 13:30:54.455141 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.455107 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-wgmgp_693f53d4-7f20-47dd-b4d4-48e57e5eb60f/openshift-state-metrics/0.log" Jul 06 13:30:54.505615 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.505584 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/prometheus/0.log" Jul 06 13:30:54.532683 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.532651 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/config-reloader/0.log" Jul 06 13:30:54.561730 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.561707 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/thanos-sidecar/0.log" Jul 06 13:30:54.589388 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.589364 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/kube-rbac-proxy-web/0.log" Jul 06 13:30:54.631822 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.631800 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/kube-rbac-proxy/0.log" Jul 06 13:30:54.663243 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.663157 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/kube-rbac-proxy-thanos/0.log" Jul 06 13:30:54.700497 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.700469 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1c6ecfb7-3fda-4067-9f5b-97cc2222b28f/init-config-reloader/0.log" Jul 06 13:30:54.794440 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.794411 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-fjd6k_ff727b66-01c9-4df5-bac4-14b28594babe/prometheus-operator-admission-webhook/0.log" Jul 06 13:30:54.827069 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.827042 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf5477557-d76bd_10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa/telemeter-client/0.log" Jul 06 13:30:54.854759 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.854731 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf5477557-d76bd_10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa/reload/0.log" Jul 06 13:30:54.891722 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:54.891690 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf5477557-d76bd_10521b1b-6ef5-4d1d-9e57-b0285cfbf1fa/kube-rbac-proxy/0.log" Jul 06 13:30:55.458394 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.458363 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2"] Jul 06 13:30:55.462543 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.462513 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.471531 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.471501 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2"] Jul 06 13:30:55.584684 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.584654 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-sys\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.584878 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.584773 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2k99\" (UniqueName: \"kubernetes.io/projected/a697a045-96fa-40f8-8337-af47c7c2b194-kube-api-access-g2k99\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.584878 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.584808 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-podres\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.584878 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.584844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-proc\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.585054 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.584885 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-lib-modules\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.686384 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.686345 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2k99\" (UniqueName: \"kubernetes.io/projected/a697a045-96fa-40f8-8337-af47c7c2b194-kube-api-access-g2k99\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.686632 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.686615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-podres\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.686763 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.686750 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-proc\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.686877 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.686852 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-proc\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.686934 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.686798 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-podres\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.687002 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.686989 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-lib-modules\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.687161 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.687150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-sys\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.687413 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.687396 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-sys\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.687507 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.687048 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a697a045-96fa-40f8-8337-af47c7c2b194-lib-modules\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.697030 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.697001 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2k99\" (UniqueName: \"kubernetes.io/projected/a697a045-96fa-40f8-8337-af47c7c2b194-kube-api-access-g2k99\") pod \"perf-node-gather-daemonset-5xgf2\" (UID: \"a697a045-96fa-40f8-8337-af47c7c2b194\") " pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.775980 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.775901 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:55.960619 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:55.960593 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2"] Jul 06 13:30:55.963584 ip-10-0-136-21 kubenswrapper[2559]: W0706 13:30:55.963554 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda697a045_96fa_40f8_8337_af47c7c2b194.slice/crio-638b3f035b62672251c61d55e43390957b28de4d8e749847b7d69a5b503c19ca WatchSource:0}: Error finding container 638b3f035b62672251c61d55e43390957b28de4d8e749847b7d69a5b503c19ca: Status 404 returned error can't find the container with id 638b3f035b62672251c61d55e43390957b28de4d8e749847b7d69a5b503c19ca Jul 06 13:30:56.327577 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.327509 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-j4bnc_7d68ba76-1b3b-4f6d-8404-996d7e89a84a/networking-console-plugin/0.log" Jul 06 13:30:56.906132 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.906093 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" event={"ID":"a697a045-96fa-40f8-8337-af47c7c2b194","Type":"ContainerStarted","Data":"f2088f5fa9191ac914e02d1df536d040195abd86642b52a13bd10d33478f460e"} Jul 06 13:30:56.906619 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.906158 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" event={"ID":"a697a045-96fa-40f8-8337-af47c7c2b194","Type":"ContainerStarted","Data":"638b3f035b62672251c61d55e43390957b28de4d8e749847b7d69a5b503c19ca"} Jul 06 13:30:56.906913 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.906887 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:30:56.908733 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.908710 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/1.log" Jul 06 13:30:56.914597 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.914562 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v9vm9_9cbacad8-eb42-4efc-81c1-a49709150841/console-operator/2.log" Jul 06 13:30:56.928458 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:56.928416 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" podStartSLOduration=1.92840323 podStartE2EDuration="1.92840323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 13:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 13:30:56.925916868 +0000 UTC m=+2657.007357895" watchObservedRunningTime="2026-07-06 13:30:56.92840323 +0000 UTC m=+2657.009844280" Jul 06 13:30:57.413926 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:57.413898 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-qt5jj_9810c557-1ee9-4ccf-a9b2-e961e8c834e6/download-server/0.log" Jul 06 13:30:59.295706 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:59.295679 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-stdbh_6990e881-e5ad-4208-9548-3ccabbff5f89/dns/0.log" Jul 06 13:30:59.320694 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:59.320672 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-stdbh_6990e881-e5ad-4208-9548-3ccabbff5f89/kube-rbac-proxy/0.log" Jul 06 13:30:59.346628 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:30:59.346603 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4sns4_3b1b9765-789d-4fc8-bb16-10601e312f90/dns-node-resolver/0.log" Jul 06 13:31:00.059289 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:00.059257 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6vxs7_a1cb7806-30f8-40b1-9cff-0dc679136c5a/node-ca/0.log" Jul 06 13:31:01.367033 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:01.367006 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7965d5bf57-8jf4q_99201dad-aaa5-45d7-831d-f804fb332f3c/kube-auth-proxy/0.log" Jul 06 13:31:02.425868 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:02.425837 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2ptv2_8f0c1232-a241-4386-9424-8c3011559d76/serve-healthcheck-canary/0.log" Jul 06 13:31:03.083574 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:03.083548 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-48wjq_f82becb4-8b89-4e3e-8687-eb04cc2c6b73/kube-rbac-proxy/0.log" Jul 06 13:31:03.113607 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:03.113574 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-48wjq_f82becb4-8b89-4e3e-8687-eb04cc2c6b73/exporter/0.log" Jul 06 13:31:03.143997 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:03.143967 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-48wjq_f82becb4-8b89-4e3e-8687-eb04cc2c6b73/extractor/0.log" Jul 06 13:31:03.933387 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:03.933355 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-89xhc/perf-node-gather-daemonset-5xgf2" Jul 06 13:31:06.254823 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:06.254751 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7c7f5bd644-7dc2h_44400754-fe37-4084-a232-b78c08b77b84/maas-api/0.log" Jul 06 13:31:06.268608 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:06.268583 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7965d5bf57-8jf4q_99201dad-aaa5-45d7-831d-f804fb332f3c/kube-auth-proxy/0.log" Jul 06 13:31:06.531442 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:06.531371 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-775867fb56-rbgrf_1e022082-6115-4fb1-a8c0-0a6916dd3371/manager/0.log" Jul 06 13:31:06.802124 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:06.802101 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58899bc45d-ts8m8_86603566-35bd-4275-929e-795ca6acdf60/manager/0.log" Jul 06 13:31:06.850493 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:06.850466 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58899bc45d-ts8m8_86603566-35bd-4275-929e-795ca6acdf60/copy-manifests/0.log" Jul 06 13:31:07.859107 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:07.859078 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7965d5bf57-8jf4q_99201dad-aaa5-45d7-831d-f804fb332f3c/kube-auth-proxy/0.log" Jul 06 13:31:08.410684 ip-10-0-136-21 kubenswrapper[2559]: I0706 13:31:08.410646 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67468f6576-qv2tc_9738dffd-b575-4842-bfde-f1a25e2c8b57/manager/0.log"