Jul 08 21:15:40.542086 ip-10-0-138-169 systemd[1]: Starting Kubernetes Kubelet... Jul 08 21:15:40.953862 ip-10-0-138-169 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:15:40.953862 ip-10-0-138-169 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 21:15:40.953862 ip-10-0-138-169 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:15:40.953862 ip-10-0-138-169 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 21:15:40.953862 ip-10-0-138-169 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:15:40.954802 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.954719 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 21:15:40.958975 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958958 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:15:40.958975 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958974 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958979 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958982 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958987 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958991 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958994 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.958997 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959000 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959002 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959005 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959008 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959011 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959014 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959017 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959019 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959022 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959025 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959027 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959029 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:15:40.959038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959033 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959036 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959039 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959042 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959044 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959062 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959075 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959078 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959081 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959084 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959087 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959089 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959092 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959094 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959097 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959099 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959101 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959104 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959106 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959109 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:15:40.959518 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959111 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959113 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959116 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959118 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959121 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959123 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959125 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959127 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959130 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959132 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959135 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959137 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959139 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959142 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959146 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959149 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959151 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959154 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959156 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959159 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:15:40.960014 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959162 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959164 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959166 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959169 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959171 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959174 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959177 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959179 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959183 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959185 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959188 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959190 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959193 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959195 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959198 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959200 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959203 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959206 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959209 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959211 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:15:40.960524 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959214 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959216 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959219 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959221 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959224 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959226 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959232 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959235 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959238 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959241 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959244 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959247 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959250 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959253 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959257 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959260 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959262 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959265 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:15:40.961030 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.959267 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:15:40.961535 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961522 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:15:40.961535 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961535 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961539 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961542 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961545 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961548 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961550 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961553 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961556 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961560 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961563 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961566 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961569 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961572 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961574 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961577 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961579 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961581 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961584 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961586 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:15:40.961589 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961589 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961592 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961594 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961597 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961599 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961602 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961604 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961607 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961609 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961612 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961614 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961616 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961619 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961621 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961623 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961626 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961628 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961630 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961632 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961635 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:15:40.962075 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961637 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961639 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961642 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961644 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961647 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961649 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961652 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961654 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961656 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961659 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961662 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961664 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961668 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961670 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961673 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961676 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961678 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961682 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961685 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961688 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:15:40.962587 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961691 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961693 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961696 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961699 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961702 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961704 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961707 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961709 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961712 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961714 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961717 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961719 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961722 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961724 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961726 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961729 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961732 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961739 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961741 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961744 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:15:40.963093 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961746 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961749 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961751 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961754 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961756 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961759 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961762 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961765 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961767 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961769 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961773 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961777 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961779 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961782 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961784 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961786 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961789 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961791 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.961794 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962170 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:15:40.963577 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962175 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962178 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962180 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962183 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962185 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962188 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962190 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962193 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962195 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962198 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962201 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962203 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962206 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962208 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962211 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962213 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962216 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962219 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962222 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962224 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:15:40.964077 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962227 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962230 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962232 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962234 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962237 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962240 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962242 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962244 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962247 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962250 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962252 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962255 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962257 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962259 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962262 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962265 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962268 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962274 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962278 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:15:40.964567 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962281 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962284 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962287 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962290 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962293 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962296 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962299 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962304 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962306 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962309 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962312 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962314 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962317 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962319 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962322 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962324 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962327 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962329 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962332 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:15:40.965045 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962334 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962337 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962339 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962341 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962344 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962346 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962348 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962351 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962353 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962355 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962358 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962360 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962362 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962365 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962369 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962373 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962376 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962380 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962383 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:15:40.965495 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962386 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962390 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962393 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962396 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962398 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962401 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962403 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962405 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962408 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962410 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962413 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962415 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962417 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962421 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962424 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962426 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962429 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962431 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962434 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962436 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:15:40.965955 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.962438 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962508 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962515 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962521 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962525 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962529 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962532 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962536 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962540 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962543 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962546 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962556 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962559 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962564 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962567 2578 flags.go:64] FLAG: --cgroup-root="" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962569 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962572 2578 flags.go:64] FLAG: --client-ca-file="" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962575 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962578 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962585 2578 flags.go:64] FLAG: --cluster-domain="" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962587 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962591 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962594 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962597 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 21:15:40.966546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962601 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962604 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962607 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962610 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962613 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962616 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962619 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962622 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962625 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962634 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962637 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962639 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962642 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962645 2578 flags.go:64] FLAG: --enable-server="true" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962648 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962654 2578 flags.go:64] FLAG: --event-burst="100" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962657 2578 flags.go:64] FLAG: --event-qps="50" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962660 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962662 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962665 2578 flags.go:64] FLAG: --eviction-hard="" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962674 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962677 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962680 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962683 2578 flags.go:64] FLAG: --eviction-soft="" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962685 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 21:15:40.967127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962688 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962691 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962693 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962696 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962699 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962702 2578 flags.go:64] FLAG: --feature-gates="" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962705 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962708 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962710 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962713 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962716 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962719 2578 flags.go:64] FLAG: --help="false" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962721 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-138-169.ec2.internal" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962724 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962727 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962729 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962732 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962735 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962738 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962740 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962743 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962745 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962748 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962751 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 21:15:40.967784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962754 2578 flags.go:64] FLAG: --kube-reserved="" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962757 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962759 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962762 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962766 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962769 2578 flags.go:64] FLAG: --lock-file="" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962772 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962775 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962778 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962783 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962786 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962788 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962791 2578 flags.go:64] FLAG: --logging-format="text" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962794 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962797 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962799 2578 flags.go:64] FLAG: --manifest-url="" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962802 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962806 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962809 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962826 2578 flags.go:64] FLAG: --max-pods="110" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962829 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962832 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962835 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962837 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962841 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 21:15:40.968429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962843 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962846 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962852 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962856 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962858 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962861 2578 flags.go:64] FLAG: --pod-cidr="" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962864 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962869 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962872 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962875 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962877 2578 flags.go:64] FLAG: --port="10250" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962880 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962883 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-055ee160ccc311937" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962886 2578 flags.go:64] FLAG: --qos-reserved="" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962889 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962892 2578 flags.go:64] FLAG: --register-node="true" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962895 2578 flags.go:64] FLAG: --register-with-taints="" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962898 2578 flags.go:64] FLAG: --registry-burst="10" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962900 2578 flags.go:64] FLAG: --registry-qps="5" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962903 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962906 2578 flags.go:64] FLAG: --reserved-memory="" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962909 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962912 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962915 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962917 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 21:15:40.969062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962920 2578 flags.go:64] FLAG: --runonce="false" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962922 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962925 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962928 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962930 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962933 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962936 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962939 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962942 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962944 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962947 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962949 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962952 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962955 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962958 2578 flags.go:64] FLAG: --system-cgroups="" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962963 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962968 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962970 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962973 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962978 2578 flags.go:64] FLAG: --tls-min-version="" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962981 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962984 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962986 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962989 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962992 2578 flags.go:64] FLAG: --v="2" Jul 08 21:15:40.969677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962996 2578 flags.go:64] FLAG: --version="false" Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.962999 2578 flags.go:64] FLAG: --vmodule="" Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.963002 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.963005 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963115 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963119 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963122 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963124 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963127 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963130 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963132 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963135 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963137 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963141 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963143 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963146 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963148 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963150 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963153 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963155 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:15:40.970287 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963158 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963162 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963166 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963169 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963171 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963173 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963177 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963181 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963184 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963187 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963190 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963192 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963194 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963197 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963199 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963201 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963204 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963206 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963208 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963211 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:15:40.970774 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963213 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963216 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963218 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963220 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963223 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963225 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963227 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963230 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963232 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963235 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963238 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963242 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963244 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963249 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963253 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963256 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963259 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963261 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963263 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:15:40.971313 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963266 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963268 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963271 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963273 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963276 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963278 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963281 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963283 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963286 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963288 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963290 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963293 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963295 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963297 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963300 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963302 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963305 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963307 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963309 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963312 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:15:40.971777 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963314 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963317 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963319 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963322 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963324 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963326 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963330 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963334 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963336 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963339 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963342 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963345 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963348 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963351 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963354 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963357 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963360 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963362 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963364 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:15:40.972278 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963367 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963369 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963371 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963374 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.963376 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.963381 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.970126 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.970141 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.970160 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.970166 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970219 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970224 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970227 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970230 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970233 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:15:40.972768 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970235 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970238 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970241 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970245 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970248 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970250 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970253 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970256 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970258 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970261 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970263 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970266 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970268 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970270 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970273 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970276 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970278 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970281 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970283 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970286 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:15:40.973160 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970289 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970291 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970294 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970297 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970301 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970304 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970308 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970311 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970314 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970317 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970320 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970322 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970324 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970327 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970329 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970332 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970334 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970336 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970340 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:15:40.973634 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970343 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970346 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970349 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970352 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970355 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970358 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970360 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970362 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970365 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970367 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970369 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970372 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970374 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970377 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970379 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970381 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970384 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970387 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970392 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:15:40.974098 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970395 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970398 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970401 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970403 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970406 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970408 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970410 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970413 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970416 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970418 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970421 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970423 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970425 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970428 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970430 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970433 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970435 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970438 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970440 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970442 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:15:40.974559 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970445 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970447 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970449 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970452 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970454 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970457 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970459 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970462 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970464 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970466 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970469 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970472 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970475 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970477 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970480 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:15:40.975095 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970482 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.970487 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970598 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970603 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970605 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970608 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970611 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970614 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970617 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970619 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970622 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970624 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970628 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970632 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:15:40.975466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970634 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970637 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970640 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970643 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970645 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970648 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970650 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970653 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970655 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970658 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970660 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970662 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970665 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970667 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970670 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970672 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970676 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970678 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970681 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970683 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:15:40.975830 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970686 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970688 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970691 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970693 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970695 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970698 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970700 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970702 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970705 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970707 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970709 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970712 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970714 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970716 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970719 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970721 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970723 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970726 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970728 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970731 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:15:40.976324 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970733 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970736 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970738 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970743 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970748 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970780 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970784 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970787 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970790 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970793 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970796 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970799 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970801 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970804 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970806 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970809 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970831 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970835 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970838 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:15:40.976898 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970841 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970843 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970846 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970848 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970850 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970853 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970855 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970858 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970860 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970863 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970865 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970867 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970870 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970872 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970875 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970877 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970879 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970882 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970884 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970887 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:15:40.977364 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970889 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970891 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970894 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970897 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970900 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970902 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970905 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:40.970907 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.970912 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.971009 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.973131 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.974495 2578 server.go:1019] "Starting client certificate rotation" Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.974593 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:15:40.977876 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.974627 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:15:40.978194 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.976732 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:15:40.982410 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.982383 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:15:40.993695 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.993678 2578 log.go:25] "Validated CRI v1 runtime API" Jul 08 21:15:40.995724 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.995711 2578 log.go:25] "Validated CRI v1 image API" Jul 08 21:15:40.996755 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.996730 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 21:15:40.999385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.999366 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:15:40.999461 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.999383 2578 fs.go:135] Filesystem UUIDs: map[138d7445-c47b-4c36-81d1-cf940278aa95:/dev/nvme0n1p4 18bc790e-76ee-4ee7-91bb-3bc80ab32115:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 08 21:15:40.999461 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:40.999398 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 21:15:41.004478 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.004374 2578 manager.go:216] Machine: {Timestamp:2026-07-08 21:15:41.003120068 +0000 UTC m=+0.344610416 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3127309 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b8cb663be0a1d0937477b445dd5d7 SystemUUID:ec2b8cb6-63be-0a1d-0937-477b445dd5d7 BootID:39ec4f1b-3dcd-4462-92a6-1b4acd268236 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:24:7b:a4:8b:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:24:7b:a4:8b:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:24:94:7e:0b:25 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 21:15:41.004478 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.004474 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 21:15:41.004588 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.004543 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 21:15:41.008071 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.008049 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 21:15:41.008200 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.008074 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-169.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 21:15:41.008792 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.008782 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 21:15:41.008839 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.008795 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 21:15:41.008839 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.008803 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:15:41.009602 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.009591 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:15:41.010918 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.010908 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 21:15:41.013859 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.013847 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:15:41.013992 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.013982 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 21:15:41.014027 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014014 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 08 21:15:41.014027 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014026 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 21:15:41.014081 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014037 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 21:15:41.014081 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014046 2578 kubelet.go:398] "Adding apiserver pod source" Jul 08 21:15:41.014081 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014054 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 21:15:41.014380 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014369 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:15:41.014419 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.014388 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:15:41.016490 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.016473 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 21:15:41.017497 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.017475 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5jzwt" Jul 08 21:15:41.018317 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.018302 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 21:15:41.018401 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.018332 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 21:15:41.019411 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019399 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 21:15:41.019453 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019417 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 21:15:41.019453 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019430 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 21:15:41.019453 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019438 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 21:15:41.019453 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019450 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019457 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019468 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019474 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019484 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019491 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019502 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019509 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019537 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 21:15:41.019620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.019543 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 21:15:41.022658 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.022645 2578 server.go:1268] "Started kubelet" Jul 08 21:15:41.022771 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.022721 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 21:15:41.022887 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.022756 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 21:15:41.022945 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.022793 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 21:15:41.023565 ip-10-0-138-169 systemd[1]: Started Kubernetes Kubelet. Jul 08 21:15:41.026560 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.026528 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 21:15:41.026666 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.026544 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-169.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 21:15:41.026754 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.026645 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 21:15:41.026913 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.026893 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 21:15:41.029403 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.029240 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5jzwt" Jul 08 21:15:41.030022 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.030006 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 08 21:15:41.030899 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.030875 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 21:15:41.034665 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.033313 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-169.ec2.internal.18c06d09bc74c82a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-169.ec2.internal,UID:ip-10-0-138-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-169.ec2.internal,},FirstTimestamp:2026-07-08 21:15:41.02261969 +0000 UTC m=+0.364110036,LastTimestamp:2026-07-08 21:15:41.02261969 +0000 UTC m=+0.364110036,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-169.ec2.internal,}" Jul 08 21:15:41.036599 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.036583 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 21:15:41.036689 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.036606 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 21:15:41.036689 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.036672 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.037398 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037371 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 21:15:41.037473 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037385 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 21:15:41.037473 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037415 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 21:15:41.037473 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037442 2578 factory.go:55] Registering systemd factory Jul 08 21:15:41.037473 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037457 2578 factory.go:223] Registration of the systemd container factory successfully Jul 08 21:15:41.037724 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037540 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 08 21:15:41.037724 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037549 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 08 21:15:41.037842 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037748 2578 factory.go:153] Registering CRI-O factory Jul 08 21:15:41.037842 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037759 2578 factory.go:223] Registration of the crio container factory successfully Jul 08 21:15:41.037842 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037802 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 21:15:41.037842 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037843 2578 factory.go:103] Registering Raw factory Jul 08 21:15:41.038007 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.037855 2578 manager.go:1196] Started watching for new ooms in manager Jul 08 21:15:41.038278 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.038262 2578 manager.go:349] Starting recovery of all containers Jul 08 21:15:41.040604 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.040575 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 21:15:41.040739 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.040666 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 21:15:41.048928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.048785 2578 manager.go:354] Recovery completed Jul 08 21:15:41.053437 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.053423 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.055340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055321 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.055409 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055354 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.055409 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055364 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.055874 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055860 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 21:15:41.055874 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055872 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 21:15:41.055960 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055862 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.055960 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.055888 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:15:41.056925 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.056911 2578 policy_none.go:49] "None policy: Start" Jul 08 21:15:41.056983 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.056930 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 21:15:41.056983 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.056945 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 21:15:41.057529 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.057514 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.057587 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.057540 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.057587 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.057554 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.057951 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.057941 2578 policy_none.go:47] "Start" Jul 08 21:15:41.095048 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095034 2578 manager.go:337] "Starting Device Plugin manager" Jul 08 21:15:41.095119 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.095073 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 21:15:41.095119 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095087 2578 server.go:87] "Starting device plugin registration server" Jul 08 21:15:41.095346 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095334 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 21:15:41.095403 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095349 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 21:15:41.095472 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095455 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 21:15:41.095555 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095542 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 21:15:41.095603 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.095557 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 21:15:41.096058 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.096037 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 21:15:41.096140 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.096093 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.176866 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.176836 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 21:15:41.178168 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.178152 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 21:15:41.178221 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.178198 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 21:15:41.178254 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.178223 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 21:15:41.178301 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.178262 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 21:15:41.180535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.180515 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:41.196322 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.196307 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.197118 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.197102 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.197186 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.197133 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.197186 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.197143 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.197186 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.197165 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.204794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.204757 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.204794 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.204776 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-169.ec2.internal\": node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.217789 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.217768 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.278754 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.278732 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal"] Jul 08 21:15:41.278848 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.278805 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.280573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.280557 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.280636 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.280595 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.280636 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.280610 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.281893 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.281882 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.281996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.281984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.282038 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282011 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.282613 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282598 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.282679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282626 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.282679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282626 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.282679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282642 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.282679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282663 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.282679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.282675 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.283928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.283912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.283999 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.283941 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:15:41.285299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.285285 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:15:41.285389 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.285309 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:15:41.285389 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.285321 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:15:41.303344 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.303319 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-169.ec2.internal\" not found" node="ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.306550 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.306534 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-169.ec2.internal\" not found" node="ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.318605 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.318585 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.338934 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.338915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83587e7bf75a8c3d81c2dd600a21d628-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal\" (UID: \"83587e7bf75a8c3d81c2dd600a21d628\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.338988 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.338939 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83587e7bf75a8c3d81c2dd600a21d628-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal\" (UID: \"83587e7bf75a8c3d81c2dd600a21d628\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.338988 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.338956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cacf4868144cd01ddbdab4f36e2a7a87-config\") pod \"kube-apiserver-proxy-ip-10-0-138-169.ec2.internal\" (UID: \"cacf4868144cd01ddbdab4f36e2a7a87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.419180 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.419149 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.439484 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.439464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83587e7bf75a8c3d81c2dd600a21d628-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal\" (UID: \"83587e7bf75a8c3d81c2dd600a21d628\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.439535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.439499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83587e7bf75a8c3d81c2dd600a21d628-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal\" (UID: \"83587e7bf75a8c3d81c2dd600a21d628\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.439535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.439521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cacf4868144cd01ddbdab4f36e2a7a87-config\") pod \"kube-apiserver-proxy-ip-10-0-138-169.ec2.internal\" (UID: \"cacf4868144cd01ddbdab4f36e2a7a87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.439598 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.439570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83587e7bf75a8c3d81c2dd600a21d628-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal\" (UID: \"83587e7bf75a8c3d81c2dd600a21d628\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.439630 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.439602 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cacf4868144cd01ddbdab4f36e2a7a87-config\") pod \"kube-apiserver-proxy-ip-10-0-138-169.ec2.internal\" (UID: \"cacf4868144cd01ddbdab4f36e2a7a87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.439630 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.439570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83587e7bf75a8c3d81c2dd600a21d628-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal\" (UID: \"83587e7bf75a8c3d81c2dd600a21d628\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.519898 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.519843 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.606412 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.606384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.610304 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:41.610171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" Jul 08 21:15:41.620622 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.620606 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:41.721158 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:41.721129 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:43.252774 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.252709 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-169.ec2.internal\" not found" Jul 08 21:15:43.280109 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.280089 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:43.282877 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.282854 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:43.368405 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.368382 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" Jul 08 21:15:43.377407 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.377388 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:43.378108 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.378092 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:15:43.379223 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.379209 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" Jul 08 21:15:43.391442 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.391426 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:15:43.404580 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.404563 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 21:15:43.404684 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.404653 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:15:43.404742 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.404686 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:15:43.404742 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.404719 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:15:43.404742 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.404716 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:15:43.445915 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.445891 2578 apiserver.go:52] "Watching apiserver" Jul 08 21:15:43.462580 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.462558 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 21:15:43.463590 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.463571 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-pt5h2","kube-system/global-pull-secret-syncer-zjwzd","kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt","openshift-cluster-node-tuning-operator/tuned-hmbbj","openshift-dns/node-resolver-wlhfz","openshift-multus/multus-additional-cni-plugins-nfrtf","openshift-multus/network-metrics-daemon-gpq8z","kube-system/konnectivity-agent-fvs79","openshift-image-registry/node-ca-zks8d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal","openshift-multus/multus-82fgn","openshift-network-diagnostics/network-check-target-948kh","openshift-network-operator/iptables-alerter-v6jzh"] Jul 08 21:15:43.465752 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.465716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.466754 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.466596 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:43.466754 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.466685 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:43.467789 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.467771 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 21:15:43.467887 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.467791 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.468572 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.468557 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4g5s2\"" Jul 08 21:15:43.468981 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.468957 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 21:15:43.469082 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.468990 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.469082 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.469024 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.469830 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.469816 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 21:15:43.469905 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.469832 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 21:15:43.469905 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.469832 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.470172 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.470157 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.470301 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.470284 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5ghws\"" Jul 08 21:15:43.470348 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.470305 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.470348 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.470321 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.470422 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.470406 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 21:15:43.471178 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471131 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 21:10:42 +0000 UTC" deadline="2028-04-17 01:33:50.403952658 +0000 UTC" Jul 08 21:15:43.471249 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471185 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15556h18m6.932778329s" Jul 08 21:15:43.471249 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.471347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471325 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.471398 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471359 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xjjss\"" Jul 08 21:15:43.471594 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471581 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 21:15:43.471637 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.471610 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.472416 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.472400 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.472694 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.472673 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.472784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.472751 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kvv8z\"" Jul 08 21:15:43.472852 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.472840 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.473294 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:43.473336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-cnibin\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-device-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44lmk\" (UniqueName: \"kubernetes.io/projected/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-kube-api-access-44lmk\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473617 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-system-cni-dir\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473617 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473617 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t4pg\" (UniqueName: \"kubernetes.io/projected/66556e1d-1cf1-4073-bb17-f22eb8835d92-kube-api-access-8t4pg\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473617 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7bba2a4-4551-4ce5-9db2-2b5763436710-iptables-alerter-script\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.473617 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m69h2\" (UniqueName: \"kubernetes.io/projected/e7bba2a4-4551-4ce5-9db2-2b5763436710-kube-api-access-m69h2\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-registration-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-os-release\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7bba2a4-4551-4ce5-9db2-2b5763436710-host-slash\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-socket-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473725 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tchf5\"" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-sys-fs\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.473846 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.473788 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.474793 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.474780 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 21:15:43.474953 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.474937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.475059 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.475042 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:43.475393 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.475382 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 21:15:43.475425 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.475407 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2w2lw\"" Jul 08 21:15:43.476134 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.476122 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.476703 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.476689 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.476810 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.476796 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 21:15:43.477193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.477167 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 21:15:43.477325 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.477181 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 21:15:43.477488 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.477467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:43.477559 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.477538 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:43.477625 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.477614 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 21:15:43.477676 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.477637 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2zxh6\"" Jul 08 21:15:43.477734 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.477694 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.478146 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.478130 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 21:15:43.478348 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.478336 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 21:15:43.478405 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.478349 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 21:15:43.478448 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.478407 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k6xv2\"" Jul 08 21:15:43.479269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.479254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.479614 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.479598 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:15:43.481498 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.481483 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 21:15:43.481577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.481510 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-24dcc\"" Jul 08 21:15:43.503632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.503578 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hsmv6" Jul 08 21:15:43.510886 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.510857 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hsmv6" Jul 08 21:15:43.569438 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.569413 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 21:15:43.573992 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.573970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.573992 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.573998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.574158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63a1a579-bb03-4938-bec5-a6c6231fb30b-agent-certs\") pod \"konnectivity-agent-fvs79\" (UID: \"63a1a579-bb03-4938-bec5-a6c6231fb30b\") " pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.574158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-system-cni-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.574158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-conf-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.574158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574102 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40093444-e186-40d8-95d8-f48c95daceb7-multus-daemon-config\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.574158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-ovn\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-cni-netd\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574167 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574182 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-system-cni-dir\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-modprobe-d\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysconfig\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovn-node-metrics-cert\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-system-cni-dir\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-kubelet-config\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rglqz\" (UniqueName: \"kubernetes.io/projected/0ebaf425-2624-4be4-b8e3-0676363ddf45-kube-api-access-rglqz\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:43.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574356 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zckzn\" (UniqueName: \"kubernetes.io/projected/3c83b512-6ff0-402d-85a9-44883cb68f6a-kube-api-access-zckzn\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7bba2a4-4551-4ce5-9db2-2b5763436710-iptables-alerter-script\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m69h2\" (UniqueName: \"kubernetes.io/projected/e7bba2a4-4551-4ce5-9db2-2b5763436710-kube-api-access-m69h2\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-lib-modules\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574549 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-tmp\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-sys-fs\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-var-lib-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574631 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-sys-fs\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574674 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-log-socket\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574713 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-env-overrides\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-host\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.574763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-serviceca\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c83b512-6ff0-402d-85a9-44883cb68f6a-hosts-file\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-os-release\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-k8s-cni-cncf-io\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574949 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-device-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.574973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44lmk\" (UniqueName: \"kubernetes.io/projected/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-kube-api-access-44lmk\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575012 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-device-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575028 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7bba2a4-4551-4ce5-9db2-2b5763436710-iptables-alerter-script\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4pg\" (UniqueName: \"kubernetes.io/projected/66556e1d-1cf1-4073-bb17-f22eb8835d92-kube-api-access-8t4pg\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.575233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-host\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575251 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-tuned\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-slash\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-run\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-var-lib-kubelet\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575359 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-cni-binary-copy\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575370 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-dbus\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575392 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40093444-e186-40d8-95d8-f48c95daceb7-cni-binary-copy\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysctl-d\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-multus-certs\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-etc-kubernetes\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-systemd\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dgr7\" (UniqueName: \"kubernetes.io/projected/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-kube-api-access-4dgr7\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-registration-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-os-release\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.575748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63a1a579-bb03-4938-bec5-a6c6231fb30b-konnectivity-ca\") pod \"konnectivity-agent-fvs79\" (UID: \"63a1a579-bb03-4938-bec5-a6c6231fb30b\") " pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-cni-bin\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575605 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysctl-conf\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575630 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-registration-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-systemd-units\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-node-log\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575662 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-os-release\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7bba2a4-4551-4ce5-9db2-2b5763436710-host-slash\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-systemd\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-etc-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575732 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7bba2a4-4551-4ce5-9db2-2b5763436710-host-slash\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovnkube-script-lib\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-cnibin\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-cni-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-netns\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.575885 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66556e1d-1cf1-4073-bb17-f22eb8835d92-cnibin\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.576290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-cni-multus\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-run-netns\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576075 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-cni-bin\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-cnibin\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-hostroot\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-kubernetes\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qt77\" (UniqueName: \"kubernetes.io/projected/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-kube-api-access-6qt77\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-kubelet\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-kubelet\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bj6b\" (UniqueName: \"kubernetes.io/projected/40093444-e186-40d8-95d8-f48c95daceb7-kube-api-access-5bj6b\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576251 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-sys\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-run-ovn-kubernetes\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576292 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-socket-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-socket-dir-parent\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovnkube-config\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg8c4\" (UniqueName: \"kubernetes.io/projected/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-kube-api-access-zg8c4\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.576769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576399 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-socket-dir\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.577303 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:43.577303 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c83b512-6ff0-402d-85a9-44883cb68f6a-tmp-dir\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.577303 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.576563 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66556e1d-1cf1-4073-bb17-f22eb8835d92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.580282 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.580261 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 21:15:43.580856 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.580836 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:15:43.580968 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.580861 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:15:43.580968 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.580902 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:43.581059 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.580975 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:15:44.080943549 +0000 UTC m=+1.991380947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:43.583467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.583437 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t4pg\" (UniqueName: \"kubernetes.io/projected/66556e1d-1cf1-4073-bb17-f22eb8835d92-kube-api-access-8t4pg\") pod \"multus-additional-cni-plugins-nfrtf\" (UID: \"66556e1d-1cf1-4073-bb17-f22eb8835d92\") " pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.583558 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.583467 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m69h2\" (UniqueName: \"kubernetes.io/projected/e7bba2a4-4551-4ce5-9db2-2b5763436710-kube-api-access-m69h2\") pod \"iptables-alerter-v6jzh\" (UID: \"e7bba2a4-4551-4ce5-9db2-2b5763436710\") " pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.583558 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.583479 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44lmk\" (UniqueName: \"kubernetes.io/projected/6d39a6ed-8ea1-4734-98c9-22ea33a07c77-kube-api-access-44lmk\") pod \"aws-ebs-csi-driver-node-vqtlt\" (UID: \"6d39a6ed-8ea1-4734-98c9-22ea33a07c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.618723 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.618694 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83587e7bf75a8c3d81c2dd600a21d628.slice/crio-ed4b58b01065867bae48bf28b4e17f54f6b10b645cef2a9a9ca27d3e183aff80 WatchSource:0}: Error finding container ed4b58b01065867bae48bf28b4e17f54f6b10b645cef2a9a9ca27d3e183aff80: Status 404 returned error can't find the container with id ed4b58b01065867bae48bf28b4e17f54f6b10b645cef2a9a9ca27d3e183aff80 Jul 08 21:15:43.623708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.623687 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:15:43.676703 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63a1a579-bb03-4938-bec5-a6c6231fb30b-konnectivity-ca\") pod \"konnectivity-agent-fvs79\" (UID: \"63a1a579-bb03-4938-bec5-a6c6231fb30b\") " pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-cni-bin\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysctl-conf\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-systemd-units\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-node-log\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-systemd\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676800 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-cni-bin\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.676822 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676815 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-systemd-units\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676812 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-systemd\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-etc-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-node-log\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676910 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-etc-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovnkube-script-lib\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-cni-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-netns\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.676994 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysctl-conf\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677043 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-netns\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-cni-multus\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-run-netns\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-cni-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-cni-bin\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-run-netns\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-cni-multus\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677152 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-cni-bin\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-cnibin\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.677193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-hostroot\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-kubernetes\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677205 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-cnibin\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677221 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-hostroot\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qt77\" (UniqueName: \"kubernetes.io/projected/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-kube-api-access-6qt77\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-kubelet\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-kubernetes\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-kubelet\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677319 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-kubelet\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-var-lib-kubelet\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bj6b\" (UniqueName: \"kubernetes.io/projected/40093444-e186-40d8-95d8-f48c95daceb7-kube-api-access-5bj6b\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63a1a579-bb03-4938-bec5-a6c6231fb30b-konnectivity-ca\") pod \"konnectivity-agent-fvs79\" (UID: \"63a1a579-bb03-4938-bec5-a6c6231fb30b\") " pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677364 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-sys\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-run-ovn-kubernetes\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-socket-dir-parent\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovnkube-config\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677437 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-sys\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg8c4\" (UniqueName: \"kubernetes.io/projected/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-kube-api-access-zg8c4\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.678037 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677465 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-run-ovn-kubernetes\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c83b512-6ff0-402d-85a9-44883cb68f6a-tmp-dir\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-socket-dir-parent\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63a1a579-bb03-4938-bec5-a6c6231fb30b-agent-certs\") pod \"konnectivity-agent-fvs79\" (UID: \"63a1a579-bb03-4938-bec5-a6c6231fb30b\") " pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677532 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovnkube-script-lib\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-system-cni-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677580 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-conf-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.677587 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677625 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-multus-conf-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.677648 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:44.177631821 +0000 UTC m=+2.088069217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40093444-e186-40d8-95d8-f48c95daceb7-multus-daemon-config\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677665 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-system-cni-dir\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-ovn\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-cni-netd\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-modprobe-d\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-ovn\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysconfig\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.678884 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677805 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysconfig\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovn-node-metrics-cert\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677819 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-cni-netd\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677827 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c83b512-6ff0-402d-85a9-44883cb68f6a-tmp-dir\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677841 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-kubelet-config\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rglqz\" (UniqueName: \"kubernetes.io/projected/0ebaf425-2624-4be4-b8e3-0676363ddf45-kube-api-access-rglqz\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-modprobe-d\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zckzn\" (UniqueName: \"kubernetes.io/projected/3c83b512-6ff0-402d-85a9-44883cb68f6a-kube-api-access-zckzn\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-kubelet-config\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-lib-modules\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677970 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovnkube-config\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.677985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-tmp\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-var-lib-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-log-socket\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-env-overrides\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678119 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-log-socket\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.679632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-host\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-serviceca\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-var-lib-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678181 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c83b512-6ff0-402d-85a9-44883cb68f6a-hosts-file\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678075 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-lib-modules\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40093444-e186-40d8-95d8-f48c95daceb7-multus-daemon-config\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-os-release\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-k8s-cni-cncf-io\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678329 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c83b512-6ff0-402d-85a9-44883cb68f6a-hosts-file\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678361 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-host\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-host\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-tuned\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-slash\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678449 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-k8s-cni-cncf-io\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678387 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-run-openvswitch\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-run\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-var-lib-kubelet\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678502 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-run\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-host\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678555 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-host-slash\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678591 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-env-overrides\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678609 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-var-lib-kubelet\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678450 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-os-release\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678629 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-serviceca\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.678681 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:43.678736 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:15:44.178719051 +0000 UTC m=+2.089156455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-dbus\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40093444-e186-40d8-95d8-f48c95daceb7-cni-binary-copy\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysctl-d\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-multus-certs\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-etc-kubernetes\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-systemd\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678927 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dgr7\" (UniqueName: \"kubernetes.io/projected/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-kube-api-access-4dgr7\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.680799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678964 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-sysctl-d\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.678983 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-etc-kubernetes\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.679009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40093444-e186-40d8-95d8-f48c95daceb7-host-run-multus-certs\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.679014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-systemd\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.679057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-dbus\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.679422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40093444-e186-40d8-95d8-f48c95daceb7-cni-binary-copy\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.680159 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-tmp\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.680265 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63a1a579-bb03-4938-bec5-a6c6231fb30b-agent-certs\") pod \"konnectivity-agent-fvs79\" (UID: \"63a1a579-bb03-4938-bec5-a6c6231fb30b\") " pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.680911 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-ovn-node-metrics-cert\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.681347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.680930 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-etc-tuned\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.685209 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.685187 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg8c4\" (UniqueName: \"kubernetes.io/projected/52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63-kube-api-access-zg8c4\") pod \"node-ca-zks8d\" (UID: \"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63\") " pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.685536 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.685517 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bj6b\" (UniqueName: \"kubernetes.io/projected/40093444-e186-40d8-95d8-f48c95daceb7-kube-api-access-5bj6b\") pod \"multus-82fgn\" (UID: \"40093444-e186-40d8-95d8-f48c95daceb7\") " pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.685925 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.685906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qt77\" (UniqueName: \"kubernetes.io/projected/2b4f3dd0-cc01-4f32-84ed-731e00246d7a-kube-api-access-6qt77\") pod \"tuned-hmbbj\" (UID: \"2b4f3dd0-cc01-4f32-84ed-731e00246d7a\") " pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.685979 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.685948 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zckzn\" (UniqueName: \"kubernetes.io/projected/3c83b512-6ff0-402d-85a9-44883cb68f6a-kube-api-access-zckzn\") pod \"node-resolver-wlhfz\" (UID: \"3c83b512-6ff0-402d-85a9-44883cb68f6a\") " pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.686149 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.686133 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rglqz\" (UniqueName: \"kubernetes.io/projected/0ebaf425-2624-4be4-b8e3-0676363ddf45-kube-api-access-rglqz\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:43.686733 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.686718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dgr7\" (UniqueName: \"kubernetes.io/projected/b4fa8fb4-20f2-4200-bf3c-4eabb8024214-kube-api-access-4dgr7\") pod \"ovnkube-node-pt5h2\" (UID: \"b4fa8fb4-20f2-4200-bf3c-4eabb8024214\") " pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.804245 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.804183 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" Jul 08 21:15:43.814266 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.814241 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66556e1d_1cf1_4073_bb17_f22eb8835d92.slice/crio-fa32c8e8bdb6282895d722d501df1167c7483d9e629437e76ed7f92a171e3137 WatchSource:0}: Error finding container fa32c8e8bdb6282895d722d501df1167c7483d9e629437e76ed7f92a171e3137: Status 404 returned error can't find the container with id fa32c8e8bdb6282895d722d501df1167c7483d9e629437e76ed7f92a171e3137 Jul 08 21:15:43.823942 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.823925 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6jzh" Jul 08 21:15:43.830064 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.830042 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bba2a4_4551_4ce5_9db2_2b5763436710.slice/crio-6e7a970272c20372c5ea977dd4d44f595bbf9c998f7a774e7a160fbf5d6b41a3 WatchSource:0}: Error finding container 6e7a970272c20372c5ea977dd4d44f595bbf9c998f7a774e7a160fbf5d6b41a3: Status 404 returned error can't find the container with id 6e7a970272c20372c5ea977dd4d44f595bbf9c998f7a774e7a160fbf5d6b41a3 Jul 08 21:15:43.830580 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.830563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" Jul 08 21:15:43.837038 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.837021 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d39a6ed_8ea1_4734_98c9_22ea33a07c77.slice/crio-a9c2985ccc6acf903082334d3261ea334b8aa83c98b225b937e1a2df97950109 WatchSource:0}: Error finding container a9c2985ccc6acf903082334d3261ea334b8aa83c98b225b937e1a2df97950109: Status 404 returned error can't find the container with id a9c2985ccc6acf903082334d3261ea334b8aa83c98b225b937e1a2df97950109 Jul 08 21:15:43.847597 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.847580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" Jul 08 21:15:43.853645 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.853627 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4f3dd0_cc01_4f32_84ed_731e00246d7a.slice/crio-25f6fc0ea5eaa5b5ddcf39663188fce9d5c0b6c29eb211420c16ae145ad12c55 WatchSource:0}: Error finding container 25f6fc0ea5eaa5b5ddcf39663188fce9d5c0b6c29eb211420c16ae145ad12c55: Status 404 returned error can't find the container with id 25f6fc0ea5eaa5b5ddcf39663188fce9d5c0b6c29eb211420c16ae145ad12c55 Jul 08 21:15:43.865452 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.865436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wlhfz" Jul 08 21:15:43.871566 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.871549 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c83b512_6ff0_402d_85a9_44883cb68f6a.slice/crio-361b591fb82cc6148060cafd1cd6810561a2abbac62bcdaef645b7db72b21381 WatchSource:0}: Error finding container 361b591fb82cc6148060cafd1cd6810561a2abbac62bcdaef645b7db72b21381: Status 404 returned error can't find the container with id 361b591fb82cc6148060cafd1cd6810561a2abbac62bcdaef645b7db72b21381 Jul 08 21:15:43.881930 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.881915 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:15:43.888545 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.888524 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a1a579_bb03_4938_bec5_a6c6231fb30b.slice/crio-e5ed650eeac2045cc39af94937c7c024eab82edf5ee1e360d2c47f13d8de5c55 WatchSource:0}: Error finding container e5ed650eeac2045cc39af94937c7c024eab82edf5ee1e360d2c47f13d8de5c55: Status 404 returned error can't find the container with id e5ed650eeac2045cc39af94937c7c024eab82edf5ee1e360d2c47f13d8de5c55 Jul 08 21:15:43.891592 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.891576 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:15:43.897941 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.897919 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fa8fb4_20f2_4200_bf3c_4eabb8024214.slice/crio-51bcdba2d424b3eed245577b3e50b0b80d502ea01cc45b496a4b6fd8c1967b4d WatchSource:0}: Error finding container 51bcdba2d424b3eed245577b3e50b0b80d502ea01cc45b496a4b6fd8c1967b4d: Status 404 returned error can't find the container with id 51bcdba2d424b3eed245577b3e50b0b80d502ea01cc45b496a4b6fd8c1967b4d Jul 08 21:15:43.907755 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.907729 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zks8d" Jul 08 21:15:43.914077 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:43.914061 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-82fgn" Jul 08 21:15:43.914164 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.914145 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b53d04_7d5b_4a95_ba7a_ea8ad96f1e63.slice/crio-bc607f8c2e879376938a721eb081509b5f81d732f4602b1a577a473adb87b257 WatchSource:0}: Error finding container bc607f8c2e879376938a721eb081509b5f81d732f4602b1a577a473adb87b257: Status 404 returned error can't find the container with id bc607f8c2e879376938a721eb081509b5f81d732f4602b1a577a473adb87b257 Jul 08 21:15:43.921707 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:43.921690 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40093444_e186_40d8_95d8_f48c95daceb7.slice/crio-0ef18e9fdfcfa171269e09fcae616651ce22c277a44ddf8657ba2d29444bdd02 WatchSource:0}: Error finding container 0ef18e9fdfcfa171269e09fcae616651ce22c277a44ddf8657ba2d29444bdd02: Status 404 returned error can't find the container with id 0ef18e9fdfcfa171269e09fcae616651ce22c277a44ddf8657ba2d29444bdd02 Jul 08 21:15:44.002419 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:15:44.002395 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcacf4868144cd01ddbdab4f36e2a7a87.slice/crio-4395b983a2f879c7ccda8e1f92e6c37ad46d809c095e21978d4579fcacee66bc WatchSource:0}: Error finding container 4395b983a2f879c7ccda8e1f92e6c37ad46d809c095e21978d4579fcacee66bc: Status 404 returned error can't find the container with id 4395b983a2f879c7ccda8e1f92e6c37ad46d809c095e21978d4579fcacee66bc Jul 08 21:15:44.082302 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.082235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:44.082403 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.082340 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:15:44.082403 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.082357 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:15:44.082403 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.082366 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:44.082510 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.082412 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:15:45.082398381 +0000 UTC m=+2.992835780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:44.182673 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.182646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:44.182792 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.182693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:44.182792 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.182770 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:44.182792 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.182778 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:44.182918 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.182812 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:45.182801292 +0000 UTC m=+3.093238688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:44.182918 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:44.182831 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:15:45.182817205 +0000 UTC m=+3.093254603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:44.513207 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.513130 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:10:43 +0000 UTC" deadline="2028-04-04 05:03:20.673963512 +0000 UTC" Jul 08 21:15:44.513207 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.513161 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15247h47m36.160806109s" Jul 08 21:15:44.531845 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.531821 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:44.632291 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.632231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fvs79" event={"ID":"63a1a579-bb03-4938-bec5-a6c6231fb30b","Type":"ContainerStarted","Data":"e5ed650eeac2045cc39af94937c7c024eab82edf5ee1e360d2c47f13d8de5c55"} Jul 08 21:15:44.640575 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.640544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wlhfz" event={"ID":"3c83b512-6ff0-402d-85a9-44883cb68f6a","Type":"ContainerStarted","Data":"361b591fb82cc6148060cafd1cd6810561a2abbac62bcdaef645b7db72b21381"} Jul 08 21:15:44.644446 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.644417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6jzh" event={"ID":"e7bba2a4-4551-4ce5-9db2-2b5763436710","Type":"ContainerStarted","Data":"6e7a970272c20372c5ea977dd4d44f595bbf9c998f7a774e7a160fbf5d6b41a3"} Jul 08 21:15:44.646497 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.646472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerStarted","Data":"fa32c8e8bdb6282895d722d501df1167c7483d9e629437e76ed7f92a171e3137"} Jul 08 21:15:44.649544 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.649519 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" event={"ID":"cacf4868144cd01ddbdab4f36e2a7a87","Type":"ContainerStarted","Data":"4395b983a2f879c7ccda8e1f92e6c37ad46d809c095e21978d4579fcacee66bc"} Jul 08 21:15:44.655654 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.655429 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-82fgn" event={"ID":"40093444-e186-40d8-95d8-f48c95daceb7","Type":"ContainerStarted","Data":"0ef18e9fdfcfa171269e09fcae616651ce22c277a44ddf8657ba2d29444bdd02"} Jul 08 21:15:44.668234 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.665676 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zks8d" event={"ID":"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63","Type":"ContainerStarted","Data":"bc607f8c2e879376938a721eb081509b5f81d732f4602b1a577a473adb87b257"} Jul 08 21:15:44.669909 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.669882 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"51bcdba2d424b3eed245577b3e50b0b80d502ea01cc45b496a4b6fd8c1967b4d"} Jul 08 21:15:44.681129 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.681091 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" event={"ID":"2b4f3dd0-cc01-4f32-84ed-731e00246d7a","Type":"ContainerStarted","Data":"25f6fc0ea5eaa5b5ddcf39663188fce9d5c0b6c29eb211420c16ae145ad12c55"} Jul 08 21:15:44.685448 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.685423 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" event={"ID":"6d39a6ed-8ea1-4734-98c9-22ea33a07c77","Type":"ContainerStarted","Data":"a9c2985ccc6acf903082334d3261ea334b8aa83c98b225b937e1a2df97950109"} Jul 08 21:15:44.693609 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:44.691621 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" event={"ID":"83587e7bf75a8c3d81c2dd600a21d628","Type":"ContainerStarted","Data":"ed4b58b01065867bae48bf28b4e17f54f6b10b645cef2a9a9ca27d3e183aff80"} Jul 08 21:15:45.053783 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.053753 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:45.090966 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.090356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:45.090966 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.090501 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:15:45.090966 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.090519 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:15:45.090966 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.090532 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:45.090966 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.090586 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:15:47.090568911 +0000 UTC m=+5.001006309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:45.190750 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.190708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:45.190943 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.190787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:45.191019 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.190968 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:45.191079 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.191031 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:47.191012599 +0000 UTC m=+5.101450007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:45.191709 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.191494 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:45.191709 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.191553 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:15:47.191538028 +0000 UTC m=+5.101975423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:45.513918 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.513835 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:10:43 +0000 UTC" deadline="2028-04-25 10:23:20.819007566 +0000 UTC" Jul 08 21:15:45.513918 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.513916 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15757h7m35.305095951s" Jul 08 21:15:45.610537 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.610456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:45.610682 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.610593 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:45.611141 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.610996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:45.611141 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.611088 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:45.611342 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.611154 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:45.611342 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:45.611219 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:45.723737 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.723708 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:45.746883 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:45.746840 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:15:47.105215 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:47.104405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:47.105215 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.104763 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:15:47.105215 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.104786 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:15:47.105215 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.104796 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:47.105215 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.104842 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:15:51.104827389 +0000 UTC m=+9.015264784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:47.205358 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:47.205321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:47.205521 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:47.205402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:47.205589 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.205524 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:47.205589 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.205576 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:51.205562784 +0000 UTC m=+9.116000179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:47.205700 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.205643 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:47.205700 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.205669 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:15:51.205660457 +0000 UTC m=+9.116097852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:47.610336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:47.610092 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:47.610336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:47.610129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:47.610336 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.610217 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:47.610336 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.610254 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:47.610336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:47.610266 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:47.610336 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:47.610315 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:49.609794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:49.609727 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:49.610394 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:49.609882 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:49.610394 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:49.610189 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:49.610394 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:49.610260 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:49.610394 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:49.610294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:49.610394 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:49.610344 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:51.138637 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:51.138598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:51.139076 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.138787 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:15:51.139076 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.138814 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:15:51.139076 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.138826 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:51.139076 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.138901 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:15:59.138882097 +0000 UTC m=+17.049319495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:51.239520 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:51.239483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:51.239702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:51.239562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:51.239702 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.239626 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:51.239702 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.239669 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:51.239839 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.239704 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:59.239684137 +0000 UTC m=+17.150121532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:51.239839 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.239726 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:15:59.239716814 +0000 UTC m=+17.150154212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:51.610461 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:51.610378 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:51.610595 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:51.610378 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:51.610595 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.610526 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:51.610702 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.610592 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:51.610702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:51.610378 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:51.610702 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:51.610678 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:53.609773 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:53.609735 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:53.610252 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:53.609856 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:53.610252 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:53.610244 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:53.610361 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:53.610331 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:53.610437 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:53.610387 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:53.610486 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:53.610441 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:55.610387 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:55.609521 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:55.610387 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:55.609647 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:55.610387 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:55.610081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:55.610387 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:55.610166 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:55.610387 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:55.610223 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:55.610387 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:55.610293 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:57.609471 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:57.609432 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:57.609471 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:57.609459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:57.609984 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:57.609553 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:57.609984 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:57.609561 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:15:57.609984 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:57.609671 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:57.609984 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:57.609763 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:59.202534 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:59.202493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:59.202971 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.202670 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:15:59.202971 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.202695 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:15:59.202971 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.202706 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:59.202971 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.202754 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:16:15.202741195 +0000 UTC m=+33.113178590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:15:59.303694 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:59.303661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:59.303885 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:59.303729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:59.303885 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.303828 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:59.303885 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.303829 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:59.304050 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.303903 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:16:15.303886961 +0000 UTC m=+33.214324360 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:15:59.304050 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.303918 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:15.303911531 +0000 UTC m=+33.214348926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:15:59.610361 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:59.610228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:15:59.610361 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:59.610276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:15:59.610361 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:15:59.610338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:15:59.610620 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.610354 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:15:59.610620 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.610458 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:15:59.610620 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:15:59.610537 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:01.610244 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:01.610199 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:01.610244 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:01.610245 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:01.610715 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:01.610315 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:01.610715 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:01.610325 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:01.610715 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:01.610398 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:01.610715 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:01.610492 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:03.609538 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:03.609499 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:03.609538 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:03.609537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:03.610083 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:03.609499 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:03.610083 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:03.609620 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:03.610083 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:03.609681 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:03.610083 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:03.609755 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:04.741212 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.741045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" event={"ID":"cacf4868144cd01ddbdab4f36e2a7a87","Type":"ContainerStarted","Data":"6448b5d19d4291f847b4da1e2bacbf760dc22c85419b81419b486e6fb5769b6e"} Jul 08 21:16:04.742510 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.742487 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-82fgn" event={"ID":"40093444-e186-40d8-95d8-f48c95daceb7","Type":"ContainerStarted","Data":"fe48b008da28ad2e21025f89fff31ba604e33e24c85a9f7eb6883e5444d59f53"} Jul 08 21:16:04.746692 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.746669 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"e14b81037989271343bf396420b124cabcdb06dcfb0de39455ca154b1cfd3ac8"} Jul 08 21:16:04.746783 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.746729 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"9844fb6b7e451fa16ebe629b5720026d8d72494bca77c2222cb88a9dec05425b"} Jul 08 21:16:04.746783 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.746759 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"f3e4543dc84ab759337e722ba19f3f663250884dc06abf5c7895f4c978a6158a"} Jul 08 21:16:04.746859 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.746792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"40ca95a7c4cddbeeff9bde9663615f1daa2428a7d899f888839633a8c2da0383"} Jul 08 21:16:04.747943 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.747923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" event={"ID":"2b4f3dd0-cc01-4f32-84ed-731e00246d7a","Type":"ContainerStarted","Data":"824b9617a5402c637acda8ea32522e35e9a0ac5e8f2f0694ec12e64120732a20"} Jul 08 21:16:04.754173 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.754137 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-169.ec2.internal" podStartSLOduration=21.754125439 podStartE2EDuration="21.754125439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:04.753928198 +0000 UTC m=+22.664365615" watchObservedRunningTime="2026-07-08 21:16:04.754125439 +0000 UTC m=+22.664562856" Jul 08 21:16:04.789503 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.789456 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-82fgn" podStartSLOduration=2.470412573 podStartE2EDuration="22.789436694s" totalImagesPullingTime="20.319024121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:04.789419932 +0000 UTC m=+22.699857350" watchObservedRunningTime="2026-07-08 21:16:04.789436694 +0000 UTC m=+22.699874110" Jul 08 21:16:04.789918 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:04.789893 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hmbbj" podStartSLOduration=2.557753537 podStartE2EDuration="22.789883718s" totalImagesPullingTime="20.232130181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:04.76885759 +0000 UTC m=+22.679295007" watchObservedRunningTime="2026-07-08 21:16:04.789883718 +0000 UTC m=+22.700321135" Jul 08 21:16:05.610432 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.610207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:05.610615 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.610213 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:05.610615 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:05.610458 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:05.610615 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.610257 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:05.610615 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:05.610520 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:05.610780 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:05.610627 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:05.752203 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.752168 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zks8d" event={"ID":"52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63","Type":"ContainerStarted","Data":"53d9c6e29c21514cd6530fc1f1053424fa9cb7203ac6c94539b888c6362d9d1c"} Jul 08 21:16:05.765926 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.765893 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"8a5df7e5a31512a77ed99b5b2a83d7e0f63e47ad8919388c3bcb53e3a3736918"} Jul 08 21:16:05.766074 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.765924 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zks8d" podStartSLOduration=3.970888955 podStartE2EDuration="23.765911442s" totalImagesPullingTime="19.795022487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:05.76576007 +0000 UTC m=+23.676197486" watchObservedRunningTime="2026-07-08 21:16:05.765911442 +0000 UTC m=+23.676348862" Jul 08 21:16:05.766074 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.765976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"7ef88747eb0f06fced5ae19a4862db4f16a89ff1ea15c6c350f6447d5817ffb8"} Jul 08 21:16:05.767750 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.767718 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" event={"ID":"6d39a6ed-8ea1-4734-98c9-22ea33a07c77","Type":"ContainerStarted","Data":"a5c258f0ea74b76cdb547a55b998726ab358f73d31047da1fb2da5b6f5c0b7a3"} Jul 08 21:16:05.769417 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.769391 2578 generic.go:356] "Generic (PLEG): container finished" podID="83587e7bf75a8c3d81c2dd600a21d628" containerID="d96c93f6a3181ef1cf74d94cd19678d8ce0543e9f8fd5736120bfa90d2f41327" exitCode=0 Jul 08 21:16:05.769537 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.769515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" event={"ID":"83587e7bf75a8c3d81c2dd600a21d628","Type":"ContainerDied","Data":"d96c93f6a3181ef1cf74d94cd19678d8ce0543e9f8fd5736120bfa90d2f41327"} Jul 08 21:16:05.771035 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.770997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fvs79" event={"ID":"63a1a579-bb03-4938-bec5-a6c6231fb30b","Type":"ContainerStarted","Data":"8876e2495ea5c112409260f470c1646b848486b26006a1b582cb02e8739352e4"} Jul 08 21:16:05.773350 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.773326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wlhfz" event={"ID":"3c83b512-6ff0-402d-85a9-44883cb68f6a","Type":"ContainerStarted","Data":"4acaf7dd53aa95d2f7824a32548e58e015d81876c64d841e5d7d626098495115"} Jul 08 21:16:05.775126 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.775103 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6jzh" event={"ID":"e7bba2a4-4551-4ce5-9db2-2b5763436710","Type":"ContainerStarted","Data":"15be5a7a32444803d1d02e66ff07c717146d12fbcf88f20bb0eefe4203745b69"} Jul 08 21:16:05.776757 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.776732 2578 generic.go:356] "Generic (PLEG): container finished" podID="66556e1d-1cf1-4073-bb17-f22eb8835d92" containerID="cf2a139fabe0527e18505b761cc357e1da2494a5d5e8c3a2a049cf6dc1d8c477" exitCode=0 Jul 08 21:16:05.776898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.776830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerDied","Data":"cf2a139fabe0527e18505b761cc357e1da2494a5d5e8c3a2a049cf6dc1d8c477"} Jul 08 21:16:05.797468 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.795805 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v6jzh" podStartSLOduration=3.916637525 podStartE2EDuration="23.795788917s" totalImagesPullingTime="19.879151392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:05.79475159 +0000 UTC m=+23.705189008" watchObservedRunningTime="2026-07-08 21:16:05.795788917 +0000 UTC m=+23.706226334" Jul 08 21:16:05.836589 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.836514 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wlhfz" podStartSLOduration=3.654628722 podStartE2EDuration="23.836496461s" totalImagesPullingTime="20.181867739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:05.835790542 +0000 UTC m=+23.746227962" watchObservedRunningTime="2026-07-08 21:16:05.836496461 +0000 UTC m=+23.746933879" Jul 08 21:16:05.849124 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:05.849088 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fvs79" podStartSLOduration=12.071797002 podStartE2EDuration="23.849075206s" totalImagesPullingTime="11.777278204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:05.848704743 +0000 UTC m=+23.759142160" watchObservedRunningTime="2026-07-08 21:16:05.849075206 +0000 UTC m=+23.759512623" Jul 08 21:16:06.368018 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.367993 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 21:16:06.541647 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.541493 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T21:16:06.368015147Z","UUID":"8878a80f-08be-4fcf-aa80-ee3c84af9975","Handler":null,"Name":"","Endpoint":""} Jul 08 21:16:06.543039 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.543022 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 21:16:06.543146 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.543058 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 21:16:06.780035 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.780004 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" event={"ID":"6d39a6ed-8ea1-4734-98c9-22ea33a07c77","Type":"ContainerStarted","Data":"497f71b0c3768606885d6b4a125002710a17f6dfbb6d43a3f503611b3153bd58"} Jul 08 21:16:06.781761 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.781727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" event={"ID":"83587e7bf75a8c3d81c2dd600a21d628","Type":"ContainerStarted","Data":"e27fb6a21fbfa09198590073ea4d03927119aa463dee66fa166876d0c174779f"} Jul 08 21:16:06.795835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.795766 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-169.ec2.internal" podStartSLOduration=23.795753333 podStartE2EDuration="23.795753333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:06.795527999 +0000 UTC m=+24.705965420" watchObservedRunningTime="2026-07-08 21:16:06.795753333 +0000 UTC m=+24.706190750" Jul 08 21:16:06.947164 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.947131 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:16:06.947797 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:06.947778 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:16:07.610558 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:07.610530 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:07.610740 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:07.610641 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:07.610740 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:07.610662 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:07.610740 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:07.610731 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:07.610947 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:07.610795 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:07.610947 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:07.610885 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:07.803273 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:07.803233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"081574acf14270994ebffb0066bc6226c511c4025b24398f47b351b734dbfd62"} Jul 08 21:16:07.803678 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:07.803475 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:16:07.804009 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:07.803988 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fvs79" Jul 08 21:16:08.807459 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:08.807427 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" event={"ID":"6d39a6ed-8ea1-4734-98c9-22ea33a07c77","Type":"ContainerStarted","Data":"53a1a267be4993f2818bf9c6ffa5e5f8a09222ab53200fa889b87db97683e9e7"} Jul 08 21:16:09.609892 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:09.609643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:09.609892 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:09.609646 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:09.610169 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:09.609664 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:09.610169 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:09.609972 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:09.610169 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:09.610074 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:09.610169 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:09.610152 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:09.814064 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:09.814025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" event={"ID":"b4fa8fb4-20f2-4200-bf3c-4eabb8024214","Type":"ContainerStarted","Data":"9b45940f887c89b11746c59a9d2236274f81f4caade22d50b3b65ad1079a3dc3"} Jul 08 21:16:09.838798 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:09.838755 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" podStartSLOduration=7.64082322 podStartE2EDuration="27.838740312s" totalImagesPullingTime="20.197917092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:09.837278681 +0000 UTC m=+27.747716098" watchObservedRunningTime="2026-07-08 21:16:09.838740312 +0000 UTC m=+27.749177728" Jul 08 21:16:09.838969 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:09.838948 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vqtlt" podStartSLOduration=4.740152239 podStartE2EDuration="27.838943629s" totalImagesPullingTime="23.09879139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:08.823673653 +0000 UTC m=+26.734111070" watchObservedRunningTime="2026-07-08 21:16:09.838943629 +0000 UTC m=+27.749381046" Jul 08 21:16:10.816956 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:10.816915 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:16:10.817476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:10.817020 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:16:10.817476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:10.817054 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:16:10.833601 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:10.833520 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:16:10.833903 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:10.833884 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:16:11.439055 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:11.439018 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zjwzd"] Jul 08 21:16:11.439238 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:11.439169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:11.439312 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:11.439285 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:11.442331 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:11.442280 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-948kh"] Jul 08 21:16:11.442441 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:11.442400 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:11.442527 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:11.442506 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:11.443996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:11.443967 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gpq8z"] Jul 08 21:16:11.444115 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:11.444075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:11.444215 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:11.444178 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:12.820900 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:12.820707 2578 generic.go:356] "Generic (PLEG): container finished" podID="66556e1d-1cf1-4073-bb17-f22eb8835d92" containerID="f9d675d65cc01a54aea87292627d56d8147d5e4efdfb82578e739005478b30c0" exitCode=0 Jul 08 21:16:12.821339 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:12.820792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerDied","Data":"f9d675d65cc01a54aea87292627d56d8147d5e4efdfb82578e739005478b30c0"} Jul 08 21:16:13.610040 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:13.610003 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:13.610212 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:13.610020 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:13.610212 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:13.610094 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:13.610212 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:13.610123 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:13.610212 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:13.610198 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:13.610361 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:13.610252 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:15.229542 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.229517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:15.229799 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.229678 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:16:15.229799 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.229696 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:16:15.229799 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.229708 2578 projected.go:196] Error preparing data for projected volume kube-api-access-2pdpj for pod openshift-network-diagnostics/network-check-target-948kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:16:15.229799 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.229754 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj podName:68e171e2-52f4-40b9-b8de-db0997a41eab nodeName:}" failed. No retries permitted until 2026-07-08 21:16:47.22974131 +0000 UTC m=+65.140178709 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pdpj" (UniqueName: "kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj") pod "network-check-target-948kh" (UID: "68e171e2-52f4-40b9-b8de-db0997a41eab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:16:15.329929 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.329901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:15.330047 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.329943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:15.330047 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.330030 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:16:15.330047 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.330034 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:16:15.330156 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.330072 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret podName:fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e nodeName:}" failed. No retries permitted until 2026-07-08 21:16:47.330060351 +0000 UTC m=+65.240497750 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret") pod "global-pull-secret-syncer-zjwzd" (UID: "fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:16:15.330156 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.330085 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:47.330079294 +0000 UTC m=+65.240516689 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:16:15.610381 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.610340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:15.610535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.610363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:15.610535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.610465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:15.610535 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.610465 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:16:15.610535 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.610523 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-948kh" podUID="68e171e2-52f4-40b9-b8de-db0997a41eab" Jul 08 21:16:15.610666 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:15.610598 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjwzd" podUID="fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e" Jul 08 21:16:15.828772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.828739 2578 generic.go:356] "Generic (PLEG): container finished" podID="66556e1d-1cf1-4073-bb17-f22eb8835d92" containerID="76de9773dee52672fc0c4906093c776a6bc7b9821518f0ed1057411702765434" exitCode=0 Jul 08 21:16:15.828772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:15.828773 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerDied","Data":"76de9773dee52672fc0c4906093c776a6bc7b9821518f0ed1057411702765434"} Jul 08 21:16:16.833421 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.833244 2578 generic.go:356] "Generic (PLEG): container finished" podID="66556e1d-1cf1-4073-bb17-f22eb8835d92" containerID="c4fd3463ca7d8820cc30eb2f41ad4e6a97c97175372150902834d1921a285cad" exitCode=0 Jul 08 21:16:16.833743 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.833324 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerDied","Data":"c4fd3463ca7d8820cc30eb2f41ad4e6a97c97175372150902834d1921a285cad"} Jul 08 21:16:16.840623 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.840606 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-169.ec2.internal" event="NodeReady" Jul 08 21:16:16.840725 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.840715 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 21:16:16.877916 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.877896 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f54f9458-4t7dl"] Jul 08 21:16:16.880601 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.880588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:16.883090 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.883062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t7n5m\"" Jul 08 21:16:16.883090 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.883062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 21:16:16.883227 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.883091 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 21:16:16.883227 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.883106 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 21:16:16.887815 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.887795 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 21:16:16.892699 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.892677 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f54f9458-4t7dl"] Jul 08 21:16:16.897270 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.897248 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gfqn2"] Jul 08 21:16:16.900253 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.900239 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wznnp"] Jul 08 21:16:16.900375 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.900359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:16.902854 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.902834 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:16.902968 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.902934 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 21:16:16.902968 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.902948 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 21:16:16.903099 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.903084 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m6rnx\"" Jul 08 21:16:16.905224 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.905208 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 21:16:16.905406 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.905377 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 21:16:16.905505 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.905425 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ltz8\"" Jul 08 21:16:16.905568 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.905541 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 21:16:16.907996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.907977 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wznnp"] Jul 08 21:16:16.911583 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:16.911533 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gfqn2"] Jul 08 21:16:17.043562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5d640c3d-da59-4e06-880a-e1b27b2f6013-config-volume\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.043562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.043562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-trusted-ca\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d640c3d-da59-4e06-880a-e1b27b2f6013-tmp-dir\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-image-registry-private-configuration\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvldc\" (UniqueName: \"kubernetes.io/projected/0235050f-9f13-48dd-93a1-d180b0fd9a21-kube-api-access-mvldc\") pod \"ingress-canary-wznnp\" (UID: \"0235050f-9f13-48dd-93a1-d180b0fd9a21\") " pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxnl4\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-kube-api-access-rxnl4\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043765 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs2fl\" (UniqueName: \"kubernetes.io/projected/5d640c3d-da59-4e06-880a-e1b27b2f6013-kube-api-access-qs2fl\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76f0522b-2a49-474a-9195-5cc1a9df98c0-ca-trust-extracted\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0235050f-9f13-48dd-93a1-d180b0fd9a21-cert\") pod \"ingress-canary-wznnp\" (UID: \"0235050f-9f13-48dd-93a1-d180b0fd9a21\") " pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.043898 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-installation-pull-secrets\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.044202 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-certificates\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.044202 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.043942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-bound-sa-token\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.144828 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.144784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs2fl\" (UniqueName: \"kubernetes.io/projected/5d640c3d-da59-4e06-880a-e1b27b2f6013-kube-api-access-qs2fl\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.144828 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.144823 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76f0522b-2a49-474a-9195-5cc1a9df98c0-ca-trust-extracted\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145043 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.144850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0235050f-9f13-48dd-93a1-d180b0fd9a21-cert\") pod \"ingress-canary-wznnp\" (UID: \"0235050f-9f13-48dd-93a1-d180b0fd9a21\") " pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.145043 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.144913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-installation-pull-secrets\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145043 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.144964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-certificates\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145043 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.144992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-bound-sa-token\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145043 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5d640c3d-da59-4e06-880a-e1b27b2f6013-config-volume\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.145268 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.145268 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.145140 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:17.145268 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.145206 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:17.64518566 +0000 UTC m=+35.555623069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:17.145268 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145242 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76f0522b-2a49-474a-9195-5cc1a9df98c0-ca-trust-extracted\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145463 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-trusted-ca\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145463 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d640c3d-da59-4e06-880a-e1b27b2f6013-tmp-dir\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.145573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145489 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-image-registry-private-configuration\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvldc\" (UniqueName: \"kubernetes.io/projected/0235050f-9f13-48dd-93a1-d180b0fd9a21-kube-api-access-mvldc\") pod \"ingress-canary-wznnp\" (UID: \"0235050f-9f13-48dd-93a1-d180b0fd9a21\") " pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.145573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145552 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-certificates\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxnl4\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-kube-api-access-rxnl4\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.145772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145587 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5d640c3d-da59-4e06-880a-e1b27b2f6013-config-volume\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.145772 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.145636 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:17.145772 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.145650 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:17.145772 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.145708 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:17.645690477 +0000 UTC m=+35.556127889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:17.146042 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.145799 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d640c3d-da59-4e06-880a-e1b27b2f6013-tmp-dir\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.146423 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.146396 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-trusted-ca\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.149324 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.149306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-installation-pull-secrets\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.149324 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.149322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0235050f-9f13-48dd-93a1-d180b0fd9a21-cert\") pod \"ingress-canary-wznnp\" (UID: \"0235050f-9f13-48dd-93a1-d180b0fd9a21\") " pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.149432 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.149324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-image-registry-private-configuration\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.153880 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.153840 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs2fl\" (UniqueName: \"kubernetes.io/projected/5d640c3d-da59-4e06-880a-e1b27b2f6013-kube-api-access-qs2fl\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.154283 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.154260 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-bound-sa-token\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.154372 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.154333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvldc\" (UniqueName: \"kubernetes.io/projected/0235050f-9f13-48dd-93a1-d180b0fd9a21-kube-api-access-mvldc\") pod \"ingress-canary-wznnp\" (UID: \"0235050f-9f13-48dd-93a1-d180b0fd9a21\") " pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.155235 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.155214 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxnl4\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-kube-api-access-rxnl4\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.215236 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.215207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wznnp" Jul 08 21:16:17.367326 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.367295 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wznnp"] Jul 08 21:16:17.372741 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:16:17.372695 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0235050f_9f13_48dd_93a1_d180b0fd9a21.slice/crio-ade62bf5ba5c45114f9f1ac9d9c6df707550b263aa4348033f8714582b24a244 WatchSource:0}: Error finding container ade62bf5ba5c45114f9f1ac9d9c6df707550b263aa4348033f8714582b24a244: Status 404 returned error can't find the container with id ade62bf5ba5c45114f9f1ac9d9c6df707550b263aa4348033f8714582b24a244 Jul 08 21:16:17.609743 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.609678 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:17.609743 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.609689 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:17.610036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.609789 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:17.612251 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.612232 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:16:17.612336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.612233 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:16:17.612336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.612292 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:16:17.613336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.613316 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9bpbv\"" Jul 08 21:16:17.613336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.613325 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:16:17.613336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.613319 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nffz7\"" Jul 08 21:16:17.649168 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.649149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:17.649269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.649184 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:17.649315 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.649280 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:17.649315 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.649288 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:17.649315 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.649290 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:17.649410 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.649336 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:18.649322935 +0000 UTC m=+36.559760330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:17.649410 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:17.649349 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:18.649343052 +0000 UTC m=+36.559780447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:17.836222 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:17.836186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wznnp" event={"ID":"0235050f-9f13-48dd-93a1-d180b0fd9a21","Type":"ContainerStarted","Data":"ade62bf5ba5c45114f9f1ac9d9c6df707550b263aa4348033f8714582b24a244"} Jul 08 21:16:18.657212 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:18.657176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:18.657396 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:18.657240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:18.657396 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:18.657285 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:18.657396 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:18.657353 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:18.657396 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:18.657368 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:18.657396 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:18.657372 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:20.65735051 +0000 UTC m=+38.567787909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:18.657670 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:18.657414 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:20.657398616 +0000 UTC m=+38.567836016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:20.674135 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:20.673895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:20.674581 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:20.674171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:20.674581 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:20.674042 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:20.674581 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:20.674275 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:24.674249137 +0000 UTC m=+42.584686534 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:20.674581 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:20.674335 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:20.674581 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:20.674348 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:20.674581 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:20.674395 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:24.674382133 +0000 UTC m=+42.584819528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:23.853081 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:23.853046 2578 generic.go:356] "Generic (PLEG): container finished" podID="66556e1d-1cf1-4073-bb17-f22eb8835d92" containerID="18024b749adee275d85154c1b5119de709cda8b2bdeea7f6dc4b0f0082b6372b" exitCode=0 Jul 08 21:16:23.853521 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:23.853120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerDied","Data":"18024b749adee275d85154c1b5119de709cda8b2bdeea7f6dc4b0f0082b6372b"} Jul 08 21:16:23.854424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:23.854402 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wznnp" event={"ID":"0235050f-9f13-48dd-93a1-d180b0fd9a21","Type":"ContainerStarted","Data":"32f990500f182a27f5d46f56fa3704f9d7125eb501e0671ef702bf14e39ca581"} Jul 08 21:16:23.892331 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:23.892284 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wznnp" podStartSLOduration=2.048371414 podStartE2EDuration="7.892266896s" totalImagesPullingTime="5.843895482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:23.891036136 +0000 UTC m=+41.801473554" watchObservedRunningTime="2026-07-08 21:16:23.892266896 +0000 UTC m=+41.802704311" Jul 08 21:16:24.704545 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:24.704514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:24.704708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:24.704587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:24.704708 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:24.704652 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:24.704708 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:24.704662 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:24.704708 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:24.704670 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:24.704708 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:24.704709 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:32.704697598 +0000 UTC m=+50.615134993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:24.704958 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:24.704721 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:32.704715439 +0000 UTC m=+50.615152834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:24.859529 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:24.859497 2578 generic.go:356] "Generic (PLEG): container finished" podID="66556e1d-1cf1-4073-bb17-f22eb8835d92" containerID="1e36efd63baa789f6de90078c0548f28c21919565770db0a07bf96bf36fcdeeb" exitCode=0 Jul 08 21:16:24.859893 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:24.859579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerDied","Data":"1e36efd63baa789f6de90078c0548f28c21919565770db0a07bf96bf36fcdeeb"} Jul 08 21:16:25.864859 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:25.864830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" event={"ID":"66556e1d-1cf1-4073-bb17-f22eb8835d92","Type":"ContainerStarted","Data":"4c44b93dcc21179a1f74ad10762550269f5ae0a561deb1736fb718e43820710b"} Jul 08 21:16:25.890642 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:25.890599 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nfrtf" podStartSLOduration=8.471635972 podStartE2EDuration="43.8905863s" totalImagesPullingTime="35.418950328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:25.889783418 +0000 UTC m=+43.800220834" watchObservedRunningTime="2026-07-08 21:16:25.8905863 +0000 UTC m=+43.801023718" Jul 08 21:16:32.756439 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:32.756404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:32.756923 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:32.756452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:32.756923 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:32.756536 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:32.756923 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:32.756536 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:32.756923 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:32.756548 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:32.756923 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:32.756599 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:48.756582731 +0000 UTC m=+66.667020127 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:32.756923 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:32.756612 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:48.756605922 +0000 UTC m=+66.667043316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:42.839808 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:42.839776 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pt5h2" Jul 08 21:16:47.259666 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.259622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:47.262221 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.262203 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:16:47.272118 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.272099 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:16:47.284191 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.284169 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pdpj\" (UniqueName: \"kubernetes.io/projected/68e171e2-52f4-40b9-b8de-db0997a41eab-kube-api-access-2pdpj\") pod \"network-check-target-948kh\" (UID: \"68e171e2-52f4-40b9-b8de-db0997a41eab\") " pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:47.331468 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.331443 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9bpbv\"" Jul 08 21:16:47.340230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.340214 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:47.360290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.360259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:47.360406 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.360318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:16:47.363015 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.362985 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:16:47.363111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.363052 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:16:47.370930 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:47.370912 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 21:16:47.371013 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:47.370998 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:17:51.370959081 +0000 UTC m=+129.281396495 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : secret "metrics-daemon-secret" not found Jul 08 21:16:47.373034 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.373015 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e-original-pull-secret\") pod \"global-pull-secret-syncer-zjwzd\" (UID: \"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e\") " pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:47.460265 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.460236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-948kh"] Jul 08 21:16:47.463656 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:16:47.463628 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e171e2_52f4_40b9_b8de_db0997a41eab.slice/crio-2e730d09b5846736938e857d4407dc7d633f7b7c8323012b667b8919d7e208f4 WatchSource:0}: Error finding container 2e730d09b5846736938e857d4407dc7d633f7b7c8323012b667b8919d7e208f4: Status 404 returned error can't find the container with id 2e730d09b5846736938e857d4407dc7d633f7b7c8323012b667b8919d7e208f4 Jul 08 21:16:47.625488 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.625455 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjwzd" Jul 08 21:16:47.737171 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.737142 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zjwzd"] Jul 08 21:16:47.740137 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:16:47.740112 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb36b861_2ad4_4f39_8bf3_59e5f9a76e9e.slice/crio-d9847bf7fc3dd0195bf44a36174057b69beff5fa70582c4f41ba4c0e1cbaf0e9 WatchSource:0}: Error finding container d9847bf7fc3dd0195bf44a36174057b69beff5fa70582c4f41ba4c0e1cbaf0e9: Status 404 returned error can't find the container with id d9847bf7fc3dd0195bf44a36174057b69beff5fa70582c4f41ba4c0e1cbaf0e9 Jul 08 21:16:47.914565 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.914485 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zjwzd" event={"ID":"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e","Type":"ContainerStarted","Data":"d9847bf7fc3dd0195bf44a36174057b69beff5fa70582c4f41ba4c0e1cbaf0e9"} Jul 08 21:16:47.915479 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:47.915456 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-948kh" event={"ID":"68e171e2-52f4-40b9-b8de-db0997a41eab","Type":"ContainerStarted","Data":"2e730d09b5846736938e857d4407dc7d633f7b7c8323012b667b8919d7e208f4"} Jul 08 21:16:48.772690 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:48.772658 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:16:48.772690 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:48.772696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:16:48.773108 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:48.772788 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:16:48.773108 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:48.772798 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:16:48.773108 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:48.772805 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:16:48.773108 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:48.772846 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:17:20.772833276 +0000 UTC m=+98.683270671 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:16:48.773108 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:16:48.772858 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:17:20.772852426 +0000 UTC m=+98.683289821 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:16:51.928220 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:51.928186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-948kh" event={"ID":"68e171e2-52f4-40b9-b8de-db0997a41eab","Type":"ContainerStarted","Data":"478f151c4888fb4c30f69719d6f2abf21ebe8d8365051975b2c2cb5bf801e41e"} Jul 08 21:16:51.928660 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:51.928320 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:16:51.942728 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:51.942685 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-948kh" podStartSLOduration=65.980602537 podStartE2EDuration="1m9.942667744s" totalImagesPullingTime="3.962065207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:51.942480618 +0000 UTC m=+69.852918034" watchObservedRunningTime="2026-07-08 21:16:51.942667744 +0000 UTC m=+69.853105163" Jul 08 21:16:53.933726 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:53.933688 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zjwzd" event={"ID":"fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e","Type":"ContainerStarted","Data":"95f8475bd3a38c274443e0340fda4faa74ecd017b1bd033e87c41f48a3c44691"} Jul 08 21:16:53.949270 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:16:53.949226 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zjwzd" podStartSLOduration=66.797520039 podStartE2EDuration="1m11.949211696s" totalImagesPullingTime="5.151691657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:53.947401838 +0000 UTC m=+71.857839254" watchObservedRunningTime="2026-07-08 21:16:53.949211696 +0000 UTC m=+71.859649505" Jul 08 21:17:20.783963 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:17:20.783812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:17:20.783963 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:17:20.783857 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:17:20.784470 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:20.783970 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:17:20.784470 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:20.783982 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:17:20.784470 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:20.783979 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:17:20.784470 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:20.784039 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:18:24.784025277 +0000 UTC m=+162.694462671 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:17:20.784470 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:20.784050 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:18:24.784045088 +0000 UTC m=+162.694482483 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:17:22.932610 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:17:22.932576 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-948kh" Jul 08 21:17:51.400827 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:17:51.398608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:17:51.400827 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:51.398823 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 21:17:51.400827 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:17:51.398918 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs podName:0ebaf425-2624-4be4-b8e3-0676363ddf45 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:53.398898997 +0000 UTC m=+251.309336406 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs") pod "network-metrics-daemon-gpq8z" (UID: "0ebaf425-2624-4be4-b8e3-0676363ddf45") : secret "metrics-daemon-secret" not found Jul 08 21:18:19.889790 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:19.889736 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" Jul 08 21:18:19.908914 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:19.908886 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gfqn2" podUID="5d640c3d-da59-4e06-880a-e1b27b2f6013" Jul 08 21:18:20.133582 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:18:20.133555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfqn2" Jul 08 21:18:20.133730 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:18:20.133556 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:18:20.620287 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:20.620249 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-gpq8z" podUID="0ebaf425-2624-4be4-b8e3-0676363ddf45" Jul 08 21:18:24.819955 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:18:24.819918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:18:24.820401 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:18:24.819961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:18:24.820401 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:24.820078 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 21:18:24.820401 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:24.820089 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f54f9458-4t7dl: secret "image-registry-tls" not found Jul 08 21:18:24.820401 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:24.820092 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 21:18:24.820401 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:24.820153 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls podName:76f0522b-2a49-474a-9195-5cc1a9df98c0 nodeName:}" failed. No retries permitted until 2026-07-08 21:20:26.820137933 +0000 UTC m=+284.730575328 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls") pod "image-registry-7f54f9458-4t7dl" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0") : secret "image-registry-tls" not found Jul 08 21:18:24.820401 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:18:24.820165 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls podName:5d640c3d-da59-4e06-880a-e1b27b2f6013 nodeName:}" failed. No retries permitted until 2026-07-08 21:20:26.820159399 +0000 UTC m=+284.730596795 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls") pod "dns-default-gfqn2" (UID: "5d640c3d-da59-4e06-880a-e1b27b2f6013") : secret "dns-default-metrics-tls" not found Jul 08 21:18:31.609982 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:18:31.609937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:19:20.558125 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.558095 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph"] Jul 08 21:19:20.559949 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.559934 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" Jul 08 21:19:20.562165 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.562147 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vv8dq\"" Jul 08 21:19:20.565056 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.565031 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k"] Jul 08 21:19:20.566920 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.566902 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh"] Jul 08 21:19:20.567021 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.567008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.568454 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.568436 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-f85dw"] Jul 08 21:19:20.568547 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.568537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.569360 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.569342 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 21:19:20.569442 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.569368 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5757m\"" Jul 08 21:19:20.569618 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.569604 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:19:20.570120 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.570105 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 21:19:20.570183 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.570171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.570724 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.570705 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 21:19:20.570905 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.570893 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 21:19:20.571093 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.571067 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:19:20.572000 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.571983 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph"] Jul 08 21:19:20.573514 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.573499 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 21:19:20.573714 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.573696 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 21:19:20.573798 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.573735 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fjslg\"" Jul 08 21:19:20.573940 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.573922 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 21:19:20.574036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.573937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 21:19:20.574036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.573954 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 21:19:20.574475 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.574335 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zzwtr\"" Jul 08 21:19:20.574475 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.574433 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 21:19:20.578702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.578685 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k"] Jul 08 21:19:20.584586 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.584449 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 21:19:20.586932 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.586911 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh"] Jul 08 21:19:20.588046 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.588028 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-f85dw"] Jul 08 21:19:20.593178 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6df65827-dae8-46cf-a3c3-dab3652ffe06-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.593254 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckjs4\" (UniqueName: \"kubernetes.io/projected/0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc-kube-api-access-ckjs4\") pod \"network-check-source-989f8b8dc-c9zph\" (UID: \"0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" Jul 08 21:19:20.593254 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/006b5ab4-e339-4591-ba91-124fcf05b773-snapshots\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.593254 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/006b5ab4-e339-4591-ba91-124fcf05b773-serving-cert\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.593355 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006b5ab4-e339-4591-ba91-124fcf05b773-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.593397 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/006b5ab4-e339-4591-ba91-124fcf05b773-tmp\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.593431 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593412 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a990f07a-ea05-44ca-8843-571e01535039-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.593465 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glmpk\" (UniqueName: \"kubernetes.io/projected/006b5ab4-e339-4591-ba91-124fcf05b773-kube-api-access-glmpk\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.593497 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a990f07a-ea05-44ca-8843-571e01535039-config\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.593530 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006b5ab4-e339-4591-ba91-124fcf05b773-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.593562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cctg\" (UniqueName: \"kubernetes.io/projected/6df65827-dae8-46cf-a3c3-dab3652ffe06-kube-api-access-7cctg\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.593600 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6df65827-dae8-46cf-a3c3-dab3652ffe06-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.593659 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.593611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppjm4\" (UniqueName: \"kubernetes.io/projected/a990f07a-ea05-44ca-8843-571e01535039-kube-api-access-ppjm4\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.659508 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.659481 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf"] Jul 08 21:19:20.661464 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.661447 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-v852d"] Jul 08 21:19:20.661584 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.661565 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:20.663578 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.663563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.664178 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.664164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xljmn\"" Jul 08 21:19:20.664301 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.664286 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 21:19:20.664394 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.664376 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 21:19:20.666945 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.666922 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jcxw4\"" Jul 08 21:19:20.667038 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.666991 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 08 21:19:20.667110 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.666994 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:19:20.667166 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.667120 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 08 21:19:20.667225 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.667213 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 08 21:19:20.677038 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.677016 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf"] Jul 08 21:19:20.677935 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.677915 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-v852d"] Jul 08 21:19:20.678068 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.678050 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 08 21:19:20.694535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6df65827-dae8-46cf-a3c3-dab3652ffe06-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.694633 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694543 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b392e7e5-7359-4906-bec1-134f21238c3a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:20.694633 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckjs4\" (UniqueName: \"kubernetes.io/projected/0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc-kube-api-access-ckjs4\") pod \"network-check-source-989f8b8dc-c9zph\" (UID: \"0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" Jul 08 21:19:20.694633 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rjk\" (UniqueName: \"kubernetes.io/projected/32c2588b-239f-48bb-8565-cadad7a589a2-kube-api-access-94rjk\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.694633 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/006b5ab4-e339-4591-ba91-124fcf05b773-snapshots\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.694788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/006b5ab4-e339-4591-ba91-124fcf05b773-serving-cert\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.694788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:20.694788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c2588b-239f-48bb-8565-cadad7a589a2-trusted-ca\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.694788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006b5ab4-e339-4591-ba91-124fcf05b773-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.694968 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/006b5ab4-e339-4591-ba91-124fcf05b773-tmp\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.694968 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.694896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a990f07a-ea05-44ca-8843-571e01535039-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.695055 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32c2588b-239f-48bb-8565-cadad7a589a2-serving-cert\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.695117 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glmpk\" (UniqueName: \"kubernetes.io/projected/006b5ab4-e339-4591-ba91-124fcf05b773-kube-api-access-glmpk\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.695174 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a990f07a-ea05-44ca-8843-571e01535039-config\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.695174 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006b5ab4-e339-4591-ba91-124fcf05b773-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.695272 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32c2588b-239f-48bb-8565-cadad7a589a2-config\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.695272 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695219 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cctg\" (UniqueName: \"kubernetes.io/projected/6df65827-dae8-46cf-a3c3-dab3652ffe06-kube-api-access-7cctg\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.695272 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6df65827-dae8-46cf-a3c3-dab3652ffe06-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.695416 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695268 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/006b5ab4-e339-4591-ba91-124fcf05b773-snapshots\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.695416 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppjm4\" (UniqueName: \"kubernetes.io/projected/a990f07a-ea05-44ca-8843-571e01535039-kube-api-access-ppjm4\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.695416 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006b5ab4-e339-4591-ba91-124fcf05b773-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.695550 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695419 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6df65827-dae8-46cf-a3c3-dab3652ffe06-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.695855 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a990f07a-ea05-44ca-8843-571e01535039-config\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.695972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.695824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/006b5ab4-e339-4591-ba91-124fcf05b773-tmp\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.696515 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.696492 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006b5ab4-e339-4591-ba91-124fcf05b773-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.697560 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.697541 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a990f07a-ea05-44ca-8843-571e01535039-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.697651 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.697602 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/006b5ab4-e339-4591-ba91-124fcf05b773-serving-cert\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.697755 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.697738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6df65827-dae8-46cf-a3c3-dab3652ffe06-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.706627 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.706605 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppjm4\" (UniqueName: \"kubernetes.io/projected/a990f07a-ea05-44ca-8843-571e01535039-kube-api-access-ppjm4\") pod \"service-ca-operator-5d46d9ffd4-42z8k\" (UID: \"a990f07a-ea05-44ca-8843-571e01535039\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.706821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.706802 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glmpk\" (UniqueName: \"kubernetes.io/projected/006b5ab4-e339-4591-ba91-124fcf05b773-kube-api-access-glmpk\") pod \"insights-operator-5c847dcfc4-f85dw\" (UID: \"006b5ab4-e339-4591-ba91-124fcf05b773\") " pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.707740 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.707720 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckjs4\" (UniqueName: \"kubernetes.io/projected/0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc-kube-api-access-ckjs4\") pod \"network-check-source-989f8b8dc-c9zph\" (UID: \"0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" Jul 08 21:19:20.708075 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.708057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cctg\" (UniqueName: \"kubernetes.io/projected/6df65827-dae8-46cf-a3c3-dab3652ffe06-kube-api-access-7cctg\") pod \"kube-storage-version-migrator-operator-846bf668f7-t22lh\" (UID: \"6df65827-dae8-46cf-a3c3-dab3652ffe06\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.796441 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.796409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32c2588b-239f-48bb-8565-cadad7a589a2-config\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.796550 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.796485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b392e7e5-7359-4906-bec1-134f21238c3a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:20.796606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.796545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rjk\" (UniqueName: \"kubernetes.io/projected/32c2588b-239f-48bb-8565-cadad7a589a2-kube-api-access-94rjk\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.796606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.796581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:20.796717 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.796611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c2588b-239f-48bb-8565-cadad7a589a2-trusted-ca\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.796782 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:20.796720 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:19:20.796782 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.796759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32c2588b-239f-48bb-8565-cadad7a589a2-serving-cert\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.796908 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:20.796785 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert podName:b392e7e5-7359-4906-bec1-134f21238c3a nodeName:}" failed. No retries permitted until 2026-07-08 21:19:21.296767422 +0000 UTC m=+219.207204843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tk5vf" (UID: "b392e7e5-7359-4906-bec1-134f21238c3a") : secret "networking-console-plugin-cert" not found Jul 08 21:19:20.797085 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.797065 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/32c2588b-239f-48bb-8565-cadad7a589a2-config\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.797469 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.797446 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/32c2588b-239f-48bb-8565-cadad7a589a2-trusted-ca\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.797688 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.797669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b392e7e5-7359-4906-bec1-134f21238c3a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:20.799030 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.799010 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32c2588b-239f-48bb-8565-cadad7a589a2-serving-cert\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.804650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.804633 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rjk\" (UniqueName: \"kubernetes.io/projected/32c2588b-239f-48bb-8565-cadad7a589a2-kube-api-access-94rjk\") pod \"console-operator-68b757fb67-v852d\" (UID: \"32c2588b-239f-48bb-8565-cadad7a589a2\") " pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:20.868452 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.868370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" Jul 08 21:19:20.877807 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.877788 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" Jul 08 21:19:20.886466 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.886448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" Jul 08 21:19:20.893540 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.893520 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" Jul 08 21:19:20.978998 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:20.976967 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:21.028506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.028437 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph"] Jul 08 21:19:21.035462 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:21.035416 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc57eff_af7c_4cb5_8b18_c1cd9b9743bc.slice/crio-b803076f79a2022e55b7b6f2ec797943611d2e560f41e157ea0e5fdf8ccb3687 WatchSource:0}: Error finding container b803076f79a2022e55b7b6f2ec797943611d2e560f41e157ea0e5fdf8ccb3687: Status 404 returned error can't find the container with id b803076f79a2022e55b7b6f2ec797943611d2e560f41e157ea0e5fdf8ccb3687 Jul 08 21:19:21.044431 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.044376 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k"] Jul 08 21:19:21.048743 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:21.048676 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda990f07a_ea05_44ca_8843_571e01535039.slice/crio-57cd2a52f6305f0af4e2cba82bfb025029015388c3a0d79273ef8d866f99c6e1 WatchSource:0}: Error finding container 57cd2a52f6305f0af4e2cba82bfb025029015388c3a0d79273ef8d866f99c6e1: Status 404 returned error can't find the container with id 57cd2a52f6305f0af4e2cba82bfb025029015388c3a0d79273ef8d866f99c6e1 Jul 08 21:19:21.127672 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.127645 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-v852d"] Jul 08 21:19:21.130608 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:21.130577 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c2588b_239f_48bb_8565_cadad7a589a2.slice/crio-4e45eb652078532cd7b28c27da6a5dd3b5f13bb3c03c1b2afaec6dc45d94d5f7 WatchSource:0}: Error finding container 4e45eb652078532cd7b28c27da6a5dd3b5f13bb3c03c1b2afaec6dc45d94d5f7: Status 404 returned error can't find the container with id 4e45eb652078532cd7b28c27da6a5dd3b5f13bb3c03c1b2afaec6dc45d94d5f7 Jul 08 21:19:21.268236 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.268200 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-f85dw"] Jul 08 21:19:21.270611 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.270590 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh"] Jul 08 21:19:21.272603 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:21.272576 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006b5ab4_e339_4591_ba91_124fcf05b773.slice/crio-76d48e37d98c4aa573017e27267085aadfe2209fde809aaf812efae5300e8c11 WatchSource:0}: Error finding container 76d48e37d98c4aa573017e27267085aadfe2209fde809aaf812efae5300e8c11: Status 404 returned error can't find the container with id 76d48e37d98c4aa573017e27267085aadfe2209fde809aaf812efae5300e8c11 Jul 08 21:19:21.273205 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:21.273180 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df65827_dae8_46cf_a3c3_dab3652ffe06.slice/crio-703ec055a058768c52412336a60044a646d452bc9bdceb9e1bd3e4cc86de94d8 WatchSource:0}: Error finding container 703ec055a058768c52412336a60044a646d452bc9bdceb9e1bd3e4cc86de94d8: Status 404 returned error can't find the container with id 703ec055a058768c52412336a60044a646d452bc9bdceb9e1bd3e4cc86de94d8 Jul 08 21:19:21.273602 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.273574 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v852d" event={"ID":"32c2588b-239f-48bb-8565-cadad7a589a2","Type":"ContainerStarted","Data":"4e45eb652078532cd7b28c27da6a5dd3b5f13bb3c03c1b2afaec6dc45d94d5f7"} Jul 08 21:19:21.275189 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.275169 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" event={"ID":"0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc","Type":"ContainerStarted","Data":"0fb2e4a817f39b72c0d69a563093a4556798a9a78311cf8a9faf8f0013c575b3"} Jul 08 21:19:21.275259 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.275225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" event={"ID":"0cc57eff-af7c-4cb5-8b18-c1cd9b9743bc","Type":"ContainerStarted","Data":"b803076f79a2022e55b7b6f2ec797943611d2e560f41e157ea0e5fdf8ccb3687"} Jul 08 21:19:21.276385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.276366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" event={"ID":"a990f07a-ea05-44ca-8843-571e01535039","Type":"ContainerStarted","Data":"57cd2a52f6305f0af4e2cba82bfb025029015388c3a0d79273ef8d866f99c6e1"} Jul 08 21:19:21.288937 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.288900 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-c9zph" podStartSLOduration=1.288887532 podStartE2EDuration="1.288887532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:21.288427909 +0000 UTC m=+219.198865329" watchObservedRunningTime="2026-07-08 21:19:21.288887532 +0000 UTC m=+219.199324943" Jul 08 21:19:21.301861 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:21.301838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:21.301997 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:21.301982 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:19:21.302052 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:21.302033 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert podName:b392e7e5-7359-4906-bec1-134f21238c3a nodeName:}" failed. No retries permitted until 2026-07-08 21:19:22.30202068 +0000 UTC m=+220.212458075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tk5vf" (UID: "b392e7e5-7359-4906-bec1-134f21238c3a") : secret "networking-console-plugin-cert" not found Jul 08 21:19:22.280920 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:22.280834 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" event={"ID":"6df65827-dae8-46cf-a3c3-dab3652ffe06","Type":"ContainerStarted","Data":"703ec055a058768c52412336a60044a646d452bc9bdceb9e1bd3e4cc86de94d8"} Jul 08 21:19:22.282944 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:22.282896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" event={"ID":"006b5ab4-e339-4591-ba91-124fcf05b773","Type":"ContainerStarted","Data":"76d48e37d98c4aa573017e27267085aadfe2209fde809aaf812efae5300e8c11"} Jul 08 21:19:22.308589 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:22.308562 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:19:22.309150 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:22.308465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:22.309242 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:22.309166 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert podName:b392e7e5-7359-4906-bec1-134f21238c3a nodeName:}" failed. No retries permitted until 2026-07-08 21:19:24.30861584 +0000 UTC m=+222.219053242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tk5vf" (UID: "b392e7e5-7359-4906-bec1-134f21238c3a") : secret "networking-console-plugin-cert" not found Jul 08 21:19:24.326708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:24.326668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:24.327207 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:24.326842 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:19:24.327207 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:24.326934 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert podName:b392e7e5-7359-4906-bec1-134f21238c3a nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.326916096 +0000 UTC m=+226.237353495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tk5vf" (UID: "b392e7e5-7359-4906-bec1-134f21238c3a") : secret "networking-console-plugin-cert" not found Jul 08 21:19:25.294128 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.294029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" event={"ID":"a990f07a-ea05-44ca-8843-571e01535039","Type":"ContainerStarted","Data":"d31dbc7bc09929083d69fcc5fa13fdfc7e6c94837f218656d8b8bcf74fcc4370"} Jul 08 21:19:25.295754 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.295727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" event={"ID":"6df65827-dae8-46cf-a3c3-dab3652ffe06","Type":"ContainerStarted","Data":"135e9b699a42a7480446d4a59c49b2d6cfb75f57182a422a1f665b46dbe47b54"} Jul 08 21:19:25.297361 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.297340 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" event={"ID":"006b5ab4-e339-4591-ba91-124fcf05b773","Type":"ContainerStarted","Data":"0564531107f75b7cac72ca63120edb75dddade1d059e78282f6f71311b257331"} Jul 08 21:19:25.299232 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.299212 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/0.log" Jul 08 21:19:25.299336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.299252 2578 generic.go:356] "Generic (PLEG): container finished" podID="32c2588b-239f-48bb-8565-cadad7a589a2" containerID="ac97f042614a0462a1c248f340b274a59979aa4070ab77f00a275ea702475b06" exitCode=255 Jul 08 21:19:25.299336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.299300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v852d" event={"ID":"32c2588b-239f-48bb-8565-cadad7a589a2","Type":"ContainerDied","Data":"ac97f042614a0462a1c248f340b274a59979aa4070ab77f00a275ea702475b06"} Jul 08 21:19:25.299494 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.299481 2578 scope.go:117] "RemoveContainer" containerID="ac97f042614a0462a1c248f340b274a59979aa4070ab77f00a275ea702475b06" Jul 08 21:19:25.312064 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.312006 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" podStartSLOduration=1.347237374 podStartE2EDuration="5.31199057s" totalImagesPullingTime="3.964753196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:25.310676024 +0000 UTC m=+223.221113442" watchObservedRunningTime="2026-07-08 21:19:25.31199057 +0000 UTC m=+223.222427987" Jul 08 21:19:25.326317 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.326277 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" podStartSLOduration=1.5884887970000001 podStartE2EDuration="5.326262942s" totalImagesPullingTime="3.737774145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:25.325115279 +0000 UTC m=+223.235552697" watchObservedRunningTime="2026-07-08 21:19:25.326262942 +0000 UTC m=+223.236700353" Jul 08 21:19:25.362975 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:25.362894 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" podStartSLOduration=1.6232521100000001 podStartE2EDuration="5.362854059s" totalImagesPullingTime="3.739601949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:25.346753693 +0000 UTC m=+223.257191111" watchObservedRunningTime="2026-07-08 21:19:25.362854059 +0000 UTC m=+223.273291477" Jul 08 21:19:26.304357 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.304327 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/1.log" Jul 08 21:19:26.304849 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.304829 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/0.log" Jul 08 21:19:26.304939 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.304894 2578 generic.go:356] "Generic (PLEG): container finished" podID="32c2588b-239f-48bb-8565-cadad7a589a2" containerID="5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833" exitCode=255 Jul 08 21:19:26.305003 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.304981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v852d" event={"ID":"32c2588b-239f-48bb-8565-cadad7a589a2","Type":"ContainerDied","Data":"5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833"} Jul 08 21:19:26.305101 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.305087 2578 scope.go:117] "RemoveContainer" containerID="ac97f042614a0462a1c248f340b274a59979aa4070ab77f00a275ea702475b06" Jul 08 21:19:26.305407 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.305360 2578 scope.go:117] "RemoveContainer" containerID="5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833" Jul 08 21:19:26.305563 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:26.305538 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-v852d_openshift-console-operator(32c2588b-239f-48bb-8565-cadad7a589a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podUID="32c2588b-239f-48bb-8565-cadad7a589a2" Jul 08 21:19:26.520457 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.520422 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p9thh"] Jul 08 21:19:26.523683 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.523290 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.526487 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.526252 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 21:19:26.526487 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.526259 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gtvgz\"" Jul 08 21:19:26.526487 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.526334 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 21:19:26.533135 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.533113 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9thh"] Jul 08 21:19:26.545910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.545860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.546029 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.545952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-crio-socket\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.546029 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.546023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n5jc\" (UniqueName: \"kubernetes.io/projected/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-api-access-7n5jc\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.546153 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.546094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-data-volume\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.546153 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.546147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.646901 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.646848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-crio-socket\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.647064 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.646944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n5jc\" (UniqueName: \"kubernetes.io/projected/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-api-access-7n5jc\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.647064 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.646983 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-crio-socket\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.647064 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.647006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-data-volume\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.647064 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.647050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.647289 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.647088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.647289 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:26.647174 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:19:26.647289 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:26.647183 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:19:26.647289 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:26.647243 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:27.147221307 +0000 UTC m=+225.057658717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : secret "insights-runtime-extractor-tls" not found Jul 08 21:19:26.647289 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:26.647263 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:27.147252821 +0000 UTC m=+225.057690221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : configmap "kube-rbac-proxy" not found Jul 08 21:19:26.647451 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.647305 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-data-volume\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.657608 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.657584 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n5jc\" (UniqueName: \"kubernetes.io/projected/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-api-access-7n5jc\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:26.943106 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.943008 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn"] Jul 08 21:19:26.945196 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.945177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" Jul 08 21:19:26.947833 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.947811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6g59f\"" Jul 08 21:19:26.947946 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.947811 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 21:19:26.947946 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.947814 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 21:19:26.955812 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:26.955793 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn"] Jul 08 21:19:27.050198 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.050167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57t7c\" (UniqueName: \"kubernetes.io/projected/f3c61254-5d58-41d7-bb44-ab2f04cf502a-kube-api-access-57t7c\") pod \"migrator-d57d58cc9-h7jkn\" (UID: \"f3c61254-5d58-41d7-bb44-ab2f04cf502a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" Jul 08 21:19:27.150644 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.150603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:27.150644 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.150648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:27.150975 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.150668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57t7c\" (UniqueName: \"kubernetes.io/projected/f3c61254-5d58-41d7-bb44-ab2f04cf502a-kube-api-access-57t7c\") pod \"migrator-d57d58cc9-h7jkn\" (UID: \"f3c61254-5d58-41d7-bb44-ab2f04cf502a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" Jul 08 21:19:27.150975 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:27.150766 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:19:27.150975 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:27.150772 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:19:27.150975 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:27.150840 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.150821424 +0000 UTC m=+226.061258824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : configmap "kube-rbac-proxy" not found Jul 08 21:19:27.150975 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:27.150857 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.15084893 +0000 UTC m=+226.061286329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : secret "insights-runtime-extractor-tls" not found Jul 08 21:19:27.158851 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.158833 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57t7c\" (UniqueName: \"kubernetes.io/projected/f3c61254-5d58-41d7-bb44-ab2f04cf502a-kube-api-access-57t7c\") pod \"migrator-d57d58cc9-h7jkn\" (UID: \"f3c61254-5d58-41d7-bb44-ab2f04cf502a\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" Jul 08 21:19:27.254783 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.254695 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" Jul 08 21:19:27.311905 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.311809 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/1.log" Jul 08 21:19:27.312296 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.312276 2578 scope.go:117] "RemoveContainer" containerID="5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833" Jul 08 21:19:27.312522 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:27.312475 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-v852d_openshift-console-operator(32c2588b-239f-48bb-8565-cadad7a589a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podUID="32c2588b-239f-48bb-8565-cadad7a589a2" Jul 08 21:19:27.377714 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:27.377685 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn"] Jul 08 21:19:27.381152 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:27.381128 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3c61254_5d58_41d7_bb44_ab2f04cf502a.slice/crio-66096421b7b445a65b93513c0624d7b5f97ecdd9a4019a7581858a7d66b927bd WatchSource:0}: Error finding container 66096421b7b445a65b93513c0624d7b5f97ecdd9a4019a7581858a7d66b927bd: Status 404 returned error can't find the container with id 66096421b7b445a65b93513c0624d7b5f97ecdd9a4019a7581858a7d66b927bd Jul 08 21:19:28.159671 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:28.159634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:28.159671 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:28.159681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:28.160196 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:28.159768 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:19:28.160196 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:28.159795 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:19:28.160196 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:28.159834 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:30.159816533 +0000 UTC m=+228.070253929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : configmap "kube-rbac-proxy" not found Jul 08 21:19:28.160196 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:28.159851 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:30.159842436 +0000 UTC m=+228.070279832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : secret "insights-runtime-extractor-tls" not found Jul 08 21:19:28.316573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:28.316521 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" event={"ID":"f3c61254-5d58-41d7-bb44-ab2f04cf502a","Type":"ContainerStarted","Data":"66096421b7b445a65b93513c0624d7b5f97ecdd9a4019a7581858a7d66b927bd"} Jul 08 21:19:28.361232 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:28.361201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:28.361357 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:28.361305 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:19:28.361357 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:28.361353 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert podName:b392e7e5-7359-4906-bec1-134f21238c3a nodeName:}" failed. No retries permitted until 2026-07-08 21:19:36.361341097 +0000 UTC m=+234.271778493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tk5vf" (UID: "b392e7e5-7359-4906-bec1-134f21238c3a") : secret "networking-console-plugin-cert" not found Jul 08 21:19:28.725193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:28.725164 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wlhfz_3c83b512-6ff0-402d-85a9-44883cb68f6a/dns-node-resolver/0.log" Jul 08 21:19:29.320525 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:29.320490 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" event={"ID":"f3c61254-5d58-41d7-bb44-ab2f04cf502a","Type":"ContainerStarted","Data":"88885e51c5a1ffc458b857c5778e6a5f1af856c759bebab5a5b4106fe9be5da5"} Jul 08 21:19:29.320909 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:29.320567 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" event={"ID":"f3c61254-5d58-41d7-bb44-ab2f04cf502a","Type":"ContainerStarted","Data":"a17cba278d57b428a07c57d4b4eb4bf7395578d571034b8bc069ce6847e8cd55"} Jul 08 21:19:29.336184 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:29.336138 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-h7jkn" podStartSLOduration=1.966059945 podStartE2EDuration="3.336121657s" totalImagesPullingTime="1.370061712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:29.334553285 +0000 UTC m=+227.244990701" watchObservedRunningTime="2026-07-08 21:19:29.336121657 +0000 UTC m=+227.246559077" Jul 08 21:19:29.719628 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:29.719598 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zks8d_52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63/node-ca/0.log" Jul 08 21:19:30.175460 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:30.175424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:30.175640 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:30.175467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:30.175640 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:30.175558 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:19:30.175640 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:30.175573 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:19:30.175640 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:30.175624 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:34.175606965 +0000 UTC m=+232.086044360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : configmap "kube-rbac-proxy" not found Jul 08 21:19:30.175640 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:30.175642 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:34.175632446 +0000 UTC m=+232.086069842 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : secret "insights-runtime-extractor-tls" not found Jul 08 21:19:30.519723 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:30.519654 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wznnp_0235050f-9f13-48dd-93a1-d180b0fd9a21/serve-healthcheck-canary/0.log" Jul 08 21:19:30.979656 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:30.979627 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:30.979812 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:30.979675 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:30.979998 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:30.979986 2578 scope.go:117] "RemoveContainer" containerID="5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833" Jul 08 21:19:30.980149 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:30.980131 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-v852d_openshift-console-operator(32c2588b-239f-48bb-8565-cadad7a589a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podUID="32c2588b-239f-48bb-8565-cadad7a589a2" Jul 08 21:19:34.203997 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:34.203961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:34.204444 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:34.204003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:34.204444 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:34.204096 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 21:19:34.204444 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:34.204106 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 21:19:34.204444 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:34.204163 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:42.20414862 +0000 UTC m=+240.114586018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : secret "insights-runtime-extractor-tls" not found Jul 08 21:19:34.204444 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:34.204176 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm podName:a2d4bdc7-b04d-4556-96af-87e98ad7d4de nodeName:}" failed. No retries permitted until 2026-07-08 21:19:42.20416999 +0000 UTC m=+240.114607384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9thh" (UID: "a2d4bdc7-b04d-4556-96af-87e98ad7d4de") : configmap "kube-rbac-proxy" not found Jul 08 21:19:36.421548 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:36.421509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:36.421956 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:36.421660 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 21:19:36.421956 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:36.421734 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert podName:b392e7e5-7359-4906-bec1-134f21238c3a nodeName:}" failed. No retries permitted until 2026-07-08 21:19:52.421716027 +0000 UTC m=+250.332153422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-tk5vf" (UID: "b392e7e5-7359-4906-bec1-134f21238c3a") : secret "networking-console-plugin-cert" not found Jul 08 21:19:42.268211 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:42.268175 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:42.268211 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:42.268212 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:42.268700 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:42.268683 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:42.270775 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:42.270757 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2d4bdc7-b04d-4556-96af-87e98ad7d4de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9thh\" (UID: \"a2d4bdc7-b04d-4556-96af-87e98ad7d4de\") " pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:42.434694 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:42.434662 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9thh" Jul 08 21:19:42.561213 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:42.561135 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9thh"] Jul 08 21:19:42.563849 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:42.563823 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d4bdc7_b04d_4556_96af_87e98ad7d4de.slice/crio-87d0e758c64bb73a0dec08a4a88f5e6534c5c504bc99ee16befef3c9584c6b5a WatchSource:0}: Error finding container 87d0e758c64bb73a0dec08a4a88f5e6534c5c504bc99ee16befef3c9584c6b5a: Status 404 returned error can't find the container with id 87d0e758c64bb73a0dec08a4a88f5e6534c5c504bc99ee16befef3c9584c6b5a Jul 08 21:19:43.367915 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:43.367854 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9thh" event={"ID":"a2d4bdc7-b04d-4556-96af-87e98ad7d4de","Type":"ContainerStarted","Data":"87d0e758c64bb73a0dec08a4a88f5e6534c5c504bc99ee16befef3c9584c6b5a"} Jul 08 21:19:46.610266 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:46.610240 2578 scope.go:117] "RemoveContainer" containerID="5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833" Jul 08 21:19:47.382696 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.382658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9thh" event={"ID":"a2d4bdc7-b04d-4556-96af-87e98ad7d4de","Type":"ContainerStarted","Data":"ddf204a070a7333d9b5518799d805ab8bf4d649b932a180bd43b0f2cb9dd4b9c"} Jul 08 21:19:47.383990 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.383970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:19:47.384347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.384333 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/1.log" Jul 08 21:19:47.384396 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.384363 2578 generic.go:356] "Generic (PLEG): container finished" podID="32c2588b-239f-48bb-8565-cadad7a589a2" containerID="a5c6029daf1aa368624df471b1c530d3f2a100c6121a1e33c511850a0eefc234" exitCode=255 Jul 08 21:19:47.384396 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.384389 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v852d" event={"ID":"32c2588b-239f-48bb-8565-cadad7a589a2","Type":"ContainerDied","Data":"a5c6029daf1aa368624df471b1c530d3f2a100c6121a1e33c511850a0eefc234"} Jul 08 21:19:47.384482 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.384421 2578 scope.go:117] "RemoveContainer" containerID="5b5cbdcb8374b6d278226e6697fe8fc1f9c109de364b1f3fa2a8e3dfa699a833" Jul 08 21:19:47.384776 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:47.384724 2578 scope.go:117] "RemoveContainer" containerID="a5c6029daf1aa368624df471b1c530d3f2a100c6121a1e33c511850a0eefc234" Jul 08 21:19:47.384961 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:47.384932 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-v852d_openshift-console-operator(32c2588b-239f-48bb-8565-cadad7a589a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podUID="32c2588b-239f-48bb-8565-cadad7a589a2" Jul 08 21:19:48.389974 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:48.389944 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:19:48.392170 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:48.392145 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9thh" event={"ID":"a2d4bdc7-b04d-4556-96af-87e98ad7d4de","Type":"ContainerStarted","Data":"041d4b5946f6bb3bec869187fdde80deef1c301653e4a200e346468d17c0d992"} Jul 08 21:19:50.979561 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:50.979521 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:50.979965 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:50.979583 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:19:50.979965 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:50.979826 2578 scope.go:117] "RemoveContainer" containerID="a5c6029daf1aa368624df471b1c530d3f2a100c6121a1e33c511850a0eefc234" Jul 08 21:19:50.980030 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:50.979989 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-v852d_openshift-console-operator(32c2588b-239f-48bb-8565-cadad7a589a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podUID="32c2588b-239f-48bb-8565-cadad7a589a2" Jul 08 21:19:51.403097 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:51.403062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9thh" event={"ID":"a2d4bdc7-b04d-4556-96af-87e98ad7d4de","Type":"ContainerStarted","Data":"86e3b57c36d8bc62c4498ecc35ffaa2cc40494c0b2588d12fd1fd9862bcca0c3"} Jul 08 21:19:51.420300 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:51.420254 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p9thh" podStartSLOduration=17.730363565 podStartE2EDuration="25.42024082s" totalImagesPullingTime="7.689877255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:19:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:51.418572076 +0000 UTC m=+249.329009494" watchObservedRunningTime="2026-07-08 21:19:51.42024082 +0000 UTC m=+249.330678236" Jul 08 21:19:52.455793 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:52.455749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:52.458703 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:52.458677 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b392e7e5-7359-4906-bec1-134f21238c3a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-tk5vf\" (UID: \"b392e7e5-7359-4906-bec1-134f21238c3a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:52.474529 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:52.474507 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xljmn\"" Jul 08 21:19:52.482434 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:52.482412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" Jul 08 21:19:52.606380 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:52.606347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf"] Jul 08 21:19:52.609079 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:52.609058 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb392e7e5_7359_4906_bec1_134f21238c3a.slice/crio-ad61780ffb03f1463c9dce1b98a3beec131a506eb161ad351e45c8d88103f5d9 WatchSource:0}: Error finding container ad61780ffb03f1463c9dce1b98a3beec131a506eb161ad351e45c8d88103f5d9: Status 404 returned error can't find the container with id ad61780ffb03f1463c9dce1b98a3beec131a506eb161ad351e45c8d88103f5d9 Jul 08 21:19:53.411929 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:53.411885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" event={"ID":"b392e7e5-7359-4906-bec1-134f21238c3a","Type":"ContainerStarted","Data":"ad61780ffb03f1463c9dce1b98a3beec131a506eb161ad351e45c8d88103f5d9"} Jul 08 21:19:53.464553 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:53.464522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:19:53.467176 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:53.467149 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebaf425-2624-4be4-b8e3-0676363ddf45-metrics-certs\") pod \"network-metrics-daemon-gpq8z\" (UID: \"0ebaf425-2624-4be4-b8e3-0676363ddf45\") " pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:19:53.514077 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:53.514053 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nffz7\"" Jul 08 21:19:53.522038 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:53.522024 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpq8z" Jul 08 21:19:53.678754 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:53.678730 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gpq8z"] Jul 08 21:19:53.945057 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:53.944984 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebaf425_2624_4be4_b8e3_0676363ddf45.slice/crio-5d40a1747118baac189f405ace1fc4a5b9e4c266a8635e78295917a6be7717fd WatchSource:0}: Error finding container 5d40a1747118baac189f405ace1fc4a5b9e4c266a8635e78295917a6be7717fd: Status 404 returned error can't find the container with id 5d40a1747118baac189f405ace1fc4a5b9e4c266a8635e78295917a6be7717fd Jul 08 21:19:54.416597 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:54.416552 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" event={"ID":"b392e7e5-7359-4906-bec1-134f21238c3a","Type":"ContainerStarted","Data":"98532e15e024c93d4a69c7275fc388a76b6ef53230713e66636a8d9e929a3891"} Jul 08 21:19:54.417584 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:54.417563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpq8z" event={"ID":"0ebaf425-2624-4be4-b8e3-0676363ddf45","Type":"ContainerStarted","Data":"5d40a1747118baac189f405ace1fc4a5b9e4c266a8635e78295917a6be7717fd"} Jul 08 21:19:54.430852 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:54.430811 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-tk5vf" podStartSLOduration=33.069951124 podStartE2EDuration="34.430798063s" totalImagesPullingTime="1.360846939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:54.430276931 +0000 UTC m=+252.340714348" watchObservedRunningTime="2026-07-08 21:19:54.430798063 +0000 UTC m=+252.341235479" Jul 08 21:19:55.423006 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.422973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpq8z" event={"ID":"0ebaf425-2624-4be4-b8e3-0676363ddf45","Type":"ContainerStarted","Data":"331f3c80df4179dd8b4cf5f9f62d0746f18c4ffbba0dbc8228fec374072cad4d"} Jul 08 21:19:55.423372 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.423025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpq8z" event={"ID":"0ebaf425-2624-4be4-b8e3-0676363ddf45","Type":"ContainerStarted","Data":"cf5581530f0be1dc6e46a124367c1907c606050671e15cbc3d744f0fc17e66d2"} Jul 08 21:19:55.438945 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.438899 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gpq8z" podStartSLOduration=252.349601037 podStartE2EDuration="4m13.438884695s" totalImagesPullingTime="1.089283658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:55.437476282 +0000 UTC m=+253.347913699" watchObservedRunningTime="2026-07-08 21:19:55.438884695 +0000 UTC m=+253.349322102" Jul 08 21:19:55.503679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.503644 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd"] Jul 08 21:19:55.506196 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.506181 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:55.508834 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.508814 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 21:19:55.508834 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.508827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f4hh8\"" Jul 08 21:19:55.514269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.514248 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd"] Jul 08 21:19:55.583210 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.583176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed5b64d4-2970-4e73-8254-9b2cd83969e7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n92dd\" (UID: \"ed5b64d4-2970-4e73-8254-9b2cd83969e7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:55.684062 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.683988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed5b64d4-2970-4e73-8254-9b2cd83969e7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n92dd\" (UID: \"ed5b64d4-2970-4e73-8254-9b2cd83969e7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:55.686586 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.686560 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed5b64d4-2970-4e73-8254-9b2cd83969e7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n92dd\" (UID: \"ed5b64d4-2970-4e73-8254-9b2cd83969e7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:55.815585 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.815541 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:55.937642 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:55.937618 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd"] Jul 08 21:19:55.940099 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:55.940073 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5b64d4_2970_4e73_8254_9b2cd83969e7.slice/crio-703879bbab67e0e02c702928afe36d1faa7a958451e2e83b5d02667c672bee26 WatchSource:0}: Error finding container 703879bbab67e0e02c702928afe36d1faa7a958451e2e83b5d02667c672bee26: Status 404 returned error can't find the container with id 703879bbab67e0e02c702928afe36d1faa7a958451e2e83b5d02667c672bee26 Jul 08 21:19:56.428484 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:56.428449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" event={"ID":"ed5b64d4-2970-4e73-8254-9b2cd83969e7","Type":"ContainerStarted","Data":"703879bbab67e0e02c702928afe36d1faa7a958451e2e83b5d02667c672bee26"} Jul 08 21:19:57.433209 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.433174 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" event={"ID":"ed5b64d4-2970-4e73-8254-9b2cd83969e7","Type":"ContainerStarted","Data":"26670284e27ac945634ea0f5e415ff89320cb5881bdb9f8ae623dd2e966912c0"} Jul 08 21:19:57.433541 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.433237 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:57.437561 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.437540 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" Jul 08 21:19:57.450292 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.450251 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n92dd" podStartSLOduration=1.067179862 podStartE2EDuration="2.450236793s" totalImagesPullingTime="1.383056931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:57.448897412 +0000 UTC m=+255.359334826" watchObservedRunningTime="2026-07-08 21:19:57.450236793 +0000 UTC m=+255.360674210" Jul 08 21:19:57.560371 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.560346 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z"] Jul 08 21:19:57.562474 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.562459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.565103 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.565081 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 21:19:57.565218 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.565115 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 21:19:57.565218 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.565158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 21:19:57.565218 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.565125 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 21:19:57.565382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.565371 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 21:19:57.565434 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.565391 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5d7xs\"" Jul 08 21:19:57.570683 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.570661 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z"] Jul 08 21:19:57.699400 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.699375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.699530 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.699405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bc8804c-a59d-4446-9e53-b4fbbded9d82-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.699530 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.699443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62tgb\" (UniqueName: \"kubernetes.io/projected/2bc8804c-a59d-4446-9e53-b4fbbded9d82-kube-api-access-62tgb\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.699530 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.699501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.800678 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.800634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.800810 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.800709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.800810 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:57.800790 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 21:19:57.800902 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:19:57.800847 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-tls podName:2bc8804c-a59d-4446-9e53-b4fbbded9d82 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:58.300829948 +0000 UTC m=+256.211267362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-bq65z" (UID: "2bc8804c-a59d-4446-9e53-b4fbbded9d82") : secret "prometheus-operator-tls" not found Jul 08 21:19:57.800902 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.800884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bc8804c-a59d-4446-9e53-b4fbbded9d82-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.800980 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.800949 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62tgb\" (UniqueName: \"kubernetes.io/projected/2bc8804c-a59d-4446-9e53-b4fbbded9d82-kube-api-access-62tgb\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.801516 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.801494 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bc8804c-a59d-4446-9e53-b4fbbded9d82-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.803199 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.803166 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:57.809575 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:57.809557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62tgb\" (UniqueName: \"kubernetes.io/projected/2bc8804c-a59d-4446-9e53-b4fbbded9d82-kube-api-access-62tgb\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:58.305277 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:58.305244 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:58.307764 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:58.307745 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bc8804c-a59d-4446-9e53-b4fbbded9d82-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bq65z\" (UID: \"2bc8804c-a59d-4446-9e53-b4fbbded9d82\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:58.471974 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:58.471944 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" Jul 08 21:19:58.595791 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:58.595714 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z"] Jul 08 21:19:58.598292 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:19:58.598248 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc8804c_a59d_4446_9e53_b4fbbded9d82.slice/crio-b3150552dc76f1acf05a6c7ff3ef573d682c3c2d4754392cdbc3579f1cba01ff WatchSource:0}: Error finding container b3150552dc76f1acf05a6c7ff3ef573d682c3c2d4754392cdbc3579f1cba01ff: Status 404 returned error can't find the container with id b3150552dc76f1acf05a6c7ff3ef573d682c3c2d4754392cdbc3579f1cba01ff Jul 08 21:19:59.442524 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:19:59.442486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" event={"ID":"2bc8804c-a59d-4446-9e53-b4fbbded9d82","Type":"ContainerStarted","Data":"b3150552dc76f1acf05a6c7ff3ef573d682c3c2d4754392cdbc3579f1cba01ff"} Jul 08 21:20:01.452112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:01.452081 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" event={"ID":"2bc8804c-a59d-4446-9e53-b4fbbded9d82","Type":"ContainerStarted","Data":"8ae0f5d1418baa85850d7517526045ae99160b83c524f62d8a59cb06b47da48b"} Jul 08 21:20:01.452476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:01.452131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" event={"ID":"2bc8804c-a59d-4446-9e53-b4fbbded9d82","Type":"ContainerStarted","Data":"a8ca6ff00e822e80583390e2fe8d147e3591d32a7135cc40979e9540288fd1ec"} Jul 08 21:20:01.468999 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:01.468954 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bq65z" podStartSLOduration=2.649124717 podStartE2EDuration="4.468940738s" totalImagesPullingTime="1.819816021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:01.468859576 +0000 UTC m=+259.379297005" watchObservedRunningTime="2026-07-08 21:20:01.468940738 +0000 UTC m=+259.379378156" Jul 08 21:20:02.925142 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.925044 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x"] Jul 08 21:20:02.928028 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.928007 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mw5xx"] Jul 08 21:20:02.928145 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.928122 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:02.930821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.930800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:02.931848 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.931828 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 21:20:02.931987 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.931829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 21:20:02.931987 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.931831 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9g2b2\"" Jul 08 21:20:02.931987 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.931832 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 21:20:02.933463 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.933446 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 21:20:02.933555 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.933476 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 21:20:02.933706 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.933691 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 21:20:02.933748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.933727 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r9fbs\"" Jul 08 21:20:02.942347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:02.942328 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x"] Jul 08 21:20:03.041652 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-textfile\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.041652 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.041860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.041860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041703 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.041860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjdt\" (UniqueName: \"kubernetes.io/projected/b8c113b8-6008-47e9-a6a7-1845b08cf79c-kube-api-access-qxjdt\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.041860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h26cs\" (UniqueName: \"kubernetes.io/projected/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-api-access-h26cs\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.041860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-tls\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.041860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8c113b8-6008-47e9-a6a7-1845b08cf79c-metrics-client-ca\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-wtmp\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.041968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-sys\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.042004 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-accelerators-collector-config\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.042112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.042026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-root\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142659 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h26cs\" (UniqueName: \"kubernetes.io/projected/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-api-access-h26cs\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.142659 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-tls\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8c113b8-6008-47e9-a6a7-1845b08cf79c-metrics-client-ca\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-wtmp\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142830 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-sys\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-accelerators-collector-config\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.142919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142920 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-root\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-wtmp\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.142965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-textfile\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjdt\" (UniqueName: \"kubernetes.io/projected/b8c113b8-6008-47e9-a6a7-1845b08cf79c-kube-api-access-qxjdt\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143307 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-textfile\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143317 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-sys\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143401 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8c113b8-6008-47e9-a6a7-1845b08cf79c-metrics-client-ca\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.143702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143472 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8c113b8-6008-47e9-a6a7-1845b08cf79c-root\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.143702 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:03.143500 2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 08 21:20:03.143702 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:03.143562 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-tls podName:3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48 nodeName:}" failed. No retries permitted until 2026-07-08 21:20:03.643545695 +0000 UTC m=+261.553983091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-n9b5x" (UID: "3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48") : secret "kube-state-metrics-tls" not found Jul 08 21:20:03.144033 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143771 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-accelerators-collector-config\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.144033 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.143806 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.144151 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.144134 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.145297 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.145273 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.146038 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.146015 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8c113b8-6008-47e9-a6a7-1845b08cf79c-node-exporter-tls\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.146380 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.145881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.152793 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.152759 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjdt\" (UniqueName: \"kubernetes.io/projected/b8c113b8-6008-47e9-a6a7-1845b08cf79c-kube-api-access-qxjdt\") pod \"node-exporter-mw5xx\" (UID: \"b8c113b8-6008-47e9-a6a7-1845b08cf79c\") " pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.153096 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.153071 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h26cs\" (UniqueName: \"kubernetes.io/projected/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-api-access-h26cs\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.244684 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.244601 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mw5xx" Jul 08 21:20:03.257696 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:03.257674 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c113b8_6008_47e9_a6a7_1845b08cf79c.slice/crio-b07d165e39d374ec28de2775924d9618b9f46dd34b1dab3d97a5372eb028b569 WatchSource:0}: Error finding container b07d165e39d374ec28de2775924d9618b9f46dd34b1dab3d97a5372eb028b569: Status 404 returned error can't find the container with id b07d165e39d374ec28de2775924d9618b9f46dd34b1dab3d97a5372eb028b569 Jul 08 21:20:03.459885 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.459839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mw5xx" event={"ID":"b8c113b8-6008-47e9-a6a7-1845b08cf79c","Type":"ContainerStarted","Data":"b07d165e39d374ec28de2775924d9618b9f46dd34b1dab3d97a5372eb028b569"} Jul 08 21:20:03.647512 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.647486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.649915 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.649888 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-n9b5x\" (UID: \"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.839099 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.839063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" Jul 08 21:20:03.975604 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.974610 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:20:03.980636 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.980613 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x"] Jul 08 21:20:03.980750 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.980714 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:03.983606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.983525 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 21:20:03.983606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.983546 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 21:20:03.983606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.983557 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kf6ln\"" Jul 08 21:20:03.984256 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984119 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 21:20:03.984256 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984135 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 21:20:03.984256 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984161 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 21:20:03.984256 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 21:20:03.984481 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984438 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 21:20:03.984481 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984461 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 21:20:03.984996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.984980 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 21:20:03.994298 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:03.994266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:20:04.151507 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-web-config\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151713 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8lr\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-kube-api-access-cg8lr\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-config-volume\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-config-out\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.151917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.152141 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.151951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8lr\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-kube-api-access-cg8lr\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-config-volume\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-config-out\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-web-config\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.253925 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.253480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.254753 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.254019 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.254753 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:04.254146 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle podName:9cea011e-7f91-4866-87b9-71cde6307807 nodeName:}" failed. No retries permitted until 2026-07-08 21:20:04.754125641 +0000 UTC m=+262.664563038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9cea011e-7f91-4866-87b9-71cde6307807") : configmap references non-existent config key: ca-bundle.crt Jul 08 21:20:04.254753 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.254224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.254753 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:04.254383 2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 08 21:20:04.254753 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:04.254430 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls podName:9cea011e-7f91-4866-87b9-71cde6307807 nodeName:}" failed. No retries permitted until 2026-07-08 21:20:04.754415609 +0000 UTC m=+262.664853005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9cea011e-7f91-4866-87b9-71cde6307807") : secret "alertmanager-main-tls" not found Jul 08 21:20:04.256683 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.256629 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-config-out\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.256792 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.256690 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.256792 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.256747 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-web-config\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.257728 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.257665 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.257728 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.257713 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-config-volume\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.257903 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.257738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.257903 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.257787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.258939 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.258916 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.269113 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.269091 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8lr\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-kube-api-access-cg8lr\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.465488 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.465448 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" event={"ID":"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48","Type":"ContainerStarted","Data":"2e5cda9ce6f4f2422359bb822e664cb23b43b853e9ca5b7a087a2becd8e6db7e"} Jul 08 21:20:04.610624 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.610606 2578 scope.go:117] "RemoveContainer" containerID="a5c6029daf1aa368624df471b1c530d3f2a100c6121a1e33c511850a0eefc234" Jul 08 21:20:04.610755 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:04.610736 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-v852d_openshift-console-operator(32c2588b-239f-48bb-8565-cadad7a589a2)\"" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podUID="32c2588b-239f-48bb-8565-cadad7a589a2" Jul 08 21:20:04.758306 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.758273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.758462 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.758316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.759150 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.759125 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.760713 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.760692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.890958 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.890923 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:20:04.896189 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.896167 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-98fbc787f-2kh9s"] Jul 08 21:20:04.899895 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.899851 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:04.902888 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.902623 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dbg96sn232c92\"" Jul 08 21:20:04.902888 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.902671 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 21:20:04.902888 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.902670 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xmhc8\"" Jul 08 21:20:04.902888 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.902623 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 21:20:04.902888 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.902680 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 21:20:04.903185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.903041 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 21:20:04.903259 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.903204 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 21:20:04.911345 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:04.911282 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-98fbc787f-2kh9s"] Jul 08 21:20:05.048331 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.048313 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:20:05.050602 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:05.050570 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cea011e_7f91_4866_87b9_71cde6307807.slice/crio-f35a1a70f1e59715c8869e08d862ae63243f72bf13136fed9a4b2ebc90a137e5 WatchSource:0}: Error finding container f35a1a70f1e59715c8869e08d862ae63243f72bf13136fed9a4b2ebc90a137e5: Status 404 returned error can't find the container with id f35a1a70f1e59715c8869e08d862ae63243f72bf13136fed9a4b2ebc90a137e5 Jul 08 21:20:05.060515 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-grpc-tls\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060624 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060624 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060790 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060619 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8wtk\" (UniqueName: \"kubernetes.io/projected/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-kube-api-access-d8wtk\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060790 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060790 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060790 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-metrics-client-ca\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.060974 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.060809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-tls\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161676 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161676 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161645 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161676 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-metrics-client-ca\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-tls\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-grpc-tls\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.161961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.161836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wtk\" (UniqueName: \"kubernetes.io/projected/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-kube-api-access-d8wtk\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.162599 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.162561 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-metrics-client-ca\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.164414 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.164376 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-tls\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.164928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.164907 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-grpc-tls\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.164928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.164921 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.165033 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.165007 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.165071 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.165031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.165071 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.165047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.169045 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.169024 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8wtk\" (UniqueName: \"kubernetes.io/projected/ea6c37a5-4460-4ab5-9bfb-709395e7ee06-kube-api-access-d8wtk\") pod \"thanos-querier-98fbc787f-2kh9s\" (UID: \"ea6c37a5-4460-4ab5-9bfb-709395e7ee06\") " pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.212057 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.212028 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:05.356788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.356611 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-98fbc787f-2kh9s"] Jul 08 21:20:05.359309 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:05.359282 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6c37a5_4460_4ab5_9bfb_709395e7ee06.slice/crio-ec7f9be8d7f1a7923ce9c1afe689b29291c75f1c9ece7a996232625ca02b95f1 WatchSource:0}: Error finding container ec7f9be8d7f1a7923ce9c1afe689b29291c75f1c9ece7a996232625ca02b95f1: Status 404 returned error can't find the container with id ec7f9be8d7f1a7923ce9c1afe689b29291c75f1c9ece7a996232625ca02b95f1 Jul 08 21:20:05.469832 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.469740 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"f35a1a70f1e59715c8869e08d862ae63243f72bf13136fed9a4b2ebc90a137e5"} Jul 08 21:20:05.470794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.470764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"ec7f9be8d7f1a7923ce9c1afe689b29291c75f1c9ece7a996232625ca02b95f1"} Jul 08 21:20:05.472277 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.472244 2578 generic.go:356] "Generic (PLEG): container finished" podID="b8c113b8-6008-47e9-a6a7-1845b08cf79c" containerID="c0603145fbc85c45a1d16aaaf18021a8aa08961a578205509e66f38abf75e290" exitCode=0 Jul 08 21:20:05.472377 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:05.472287 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mw5xx" event={"ID":"b8c113b8-6008-47e9-a6a7-1845b08cf79c","Type":"ContainerDied","Data":"c0603145fbc85c45a1d16aaaf18021a8aa08961a578205509e66f38abf75e290"} Jul 08 21:20:06.478940 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.478909 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mw5xx" event={"ID":"b8c113b8-6008-47e9-a6a7-1845b08cf79c","Type":"ContainerStarted","Data":"19a2ba68ad7ca8655e5bd3649787ebcc66f78a8105f5b8be1535a96da218dddb"} Jul 08 21:20:06.479398 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.478972 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mw5xx" event={"ID":"b8c113b8-6008-47e9-a6a7-1845b08cf79c","Type":"ContainerStarted","Data":"d68095c16ea308470d42b56082afc872e408fe4561ca96da4097d145e4b760f8"} Jul 08 21:20:06.481278 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.481179 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" event={"ID":"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48","Type":"ContainerStarted","Data":"a7fd2a25178a88e35a8795728a430a5d4db9173f484acb356c0418f17e064251"} Jul 08 21:20:06.481278 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.481233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" event={"ID":"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48","Type":"ContainerStarted","Data":"8307aff7f5e300070a2e548acd346b319a3ad912885bfe11389690531452ae18"} Jul 08 21:20:06.481278 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.481262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" event={"ID":"3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48","Type":"ContainerStarted","Data":"05f863908d93f8bac78795f909f6e38166d809fbc858b8045fb0612fd59d151e"} Jul 08 21:20:06.497806 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.497345 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mw5xx" podStartSLOduration=3.123248876 podStartE2EDuration="4.497324291s" totalImagesPullingTime="1.302085203s" totalInitContainerRuntime="71.990212ms" isStatefulPod=true podCreationTimestamp="2026-07-08 21:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:06.496530345 +0000 UTC m=+264.406967765" watchObservedRunningTime="2026-07-08 21:20:06.497324291 +0000 UTC m=+264.407761756" Jul 08 21:20:06.514188 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:06.514135 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-n9b5x" podStartSLOduration=2.423117346 podStartE2EDuration="4.514115958s" totalImagesPullingTime="2.090998612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:06.512630717 +0000 UTC m=+264.423068135" watchObservedRunningTime="2026-07-08 21:20:06.514115958 +0000 UTC m=+264.424553376" Jul 08 21:20:07.487934 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.487844 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3" exitCode=0 Jul 08 21:20:07.488357 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.487938 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3"} Jul 08 21:20:07.686972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.686937 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s"] Jul 08 21:20:07.690834 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.690813 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:07.694178 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.694139 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 21:20:07.694178 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.694146 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dfmhk\"" Jul 08 21:20:07.696925 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.696902 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s"] Jul 08 21:20:07.786663 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.786587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77314b41-10fb-4a07-b2a0-f0f466e1b40d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:07.786807 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.786711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn6ch\" (UniqueName: \"kubernetes.io/projected/77314b41-10fb-4a07-b2a0-f0f466e1b40d-kube-api-access-dn6ch\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:07.887278 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.887232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77314b41-10fb-4a07-b2a0-f0f466e1b40d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:07.887451 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.887373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn6ch\" (UniqueName: \"kubernetes.io/projected/77314b41-10fb-4a07-b2a0-f0f466e1b40d-kube-api-access-dn6ch\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:07.887451 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:07.887415 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 21:20:07.887570 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:07.887500 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77314b41-10fb-4a07-b2a0-f0f466e1b40d-monitoring-plugin-cert podName:77314b41-10fb-4a07-b2a0-f0f466e1b40d nodeName:}" failed. No retries permitted until 2026-07-08 21:20:08.387477237 +0000 UTC m=+266.297914650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/77314b41-10fb-4a07-b2a0-f0f466e1b40d-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-lbl2s" (UID: "77314b41-10fb-4a07-b2a0-f0f466e1b40d") : secret "monitoring-plugin-cert" not found Jul 08 21:20:07.896946 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:07.896920 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn6ch\" (UniqueName: \"kubernetes.io/projected/77314b41-10fb-4a07-b2a0-f0f466e1b40d-kube-api-access-dn6ch\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:08.392071 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.392036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77314b41-10fb-4a07-b2a0-f0f466e1b40d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:08.394427 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.394400 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77314b41-10fb-4a07-b2a0-f0f466e1b40d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-lbl2s\" (UID: \"77314b41-10fb-4a07-b2a0-f0f466e1b40d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:08.494584 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.494552 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"8474a03c3b4c60165e36b34aa2315067de66391191b65dc8b1e6cdfc718ebbe7"} Jul 08 21:20:08.494981 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.494660 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"d8fd4c544b6e5619ecf375c3e1345eda5f5b71495cdc1216cc7bf15bafd1e54d"} Jul 08 21:20:08.494981 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.494732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"1c7979d7d45035b1969c10e5aa2d74c166ff684feae904a9b9b255c8009ea9de"} Jul 08 21:20:08.604848 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.604821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:08.740776 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:08.740751 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s"] Jul 08 21:20:08.742990 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:08.742970 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77314b41_10fb_4a07_b2a0_f0f466e1b40d.slice/crio-354808567a698854fe3307629d6f6c48679c05496a5c97729890a88b967abfa9 WatchSource:0}: Error finding container 354808567a698854fe3307629d6f6c48679c05496a5c97729890a88b967abfa9: Status 404 returned error can't find the container with id 354808567a698854fe3307629d6f6c48679c05496a5c97729890a88b967abfa9 Jul 08 21:20:09.501292 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:09.501259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"7c25967b82d3584cb522fec0f658eedf7d1a1c36f45fb5de076f67ca0b27f350"} Jul 08 21:20:09.502499 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:09.502474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" event={"ID":"77314b41-10fb-4a07-b2a0-f0f466e1b40d","Type":"ContainerStarted","Data":"354808567a698854fe3307629d6f6c48679c05496a5c97729890a88b967abfa9"} Jul 08 21:20:10.510484 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:10.510443 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"f8cf6ed7ba1218e60af88176ae7d94a0c1197a825d21cac466b7e484aa1fae37"} Jul 08 21:20:10.510937 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:10.510522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" event={"ID":"ea6c37a5-4460-4ab5-9bfb-709395e7ee06","Type":"ContainerStarted","Data":"82d1f1a6010ab2a58b8ac7a0299738800d3b2bf45d7459a54e1be2e5a5c81938"} Jul 08 21:20:10.510937 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:10.510602 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:10.533531 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:10.533480 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" podStartSLOduration=2.688049391 podStartE2EDuration="6.533461291s" totalImagesPullingTime="3.8454119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:10.530971828 +0000 UTC m=+268.441409245" watchObservedRunningTime="2026-07-08 21:20:10.533461291 +0000 UTC m=+268.443898719" Jul 08 21:20:11.516004 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.515967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" event={"ID":"77314b41-10fb-4a07-b2a0-f0f466e1b40d","Type":"ContainerStarted","Data":"5dcda0e86fc969fa93e6b8f38b2a89e9df96db913c83dd69b8709c09e49a37da"} Jul 08 21:20:11.516425 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.516025 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:11.519796 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.519766 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5"} Jul 08 21:20:11.519928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.519824 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84"} Jul 08 21:20:11.519928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.519859 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b"} Jul 08 21:20:11.519928 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.519906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a"} Jul 08 21:20:11.520067 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.519940 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183"} Jul 08 21:20:11.520067 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.519963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerStarted","Data":"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de"} Jul 08 21:20:11.521472 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.521454 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" Jul 08 21:20:11.531708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.531658 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-lbl2s" podStartSLOduration=2.499433169 podStartE2EDuration="4.531644638s" totalImagesPullingTime="2.032211469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:11.529720188 +0000 UTC m=+269.440157605" watchObservedRunningTime="2026-07-08 21:20:11.531644638 +0000 UTC m=+269.442082085" Jul 08 21:20:11.567324 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:11.567234 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.53802895 podStartE2EDuration="8.567220998s" totalImagesPullingTime="5.014598639s" totalInitContainerRuntime="14.593409ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:11.565617175 +0000 UTC m=+269.476054592" watchObservedRunningTime="2026-07-08 21:20:11.567220998 +0000 UTC m=+269.477658417" Jul 08 21:20:16.525964 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:16.525935 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-98fbc787f-2kh9s" Jul 08 21:20:16.610077 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:16.610053 2578 scope.go:117] "RemoveContainer" containerID="a5c6029daf1aa368624df471b1c530d3f2a100c6121a1e33c511850a0eefc234" Jul 08 21:20:17.545030 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.545000 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:20:17.545459 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.545098 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-v852d" event={"ID":"32c2588b-239f-48bb-8565-cadad7a589a2","Type":"ContainerStarted","Data":"984781c21770a354e0b3e6099272021dfb5ca4ec01fef69e66984c6761f557d8"} Jul 08 21:20:17.545531 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.545497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:20:17.550407 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.550382 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-v852d" Jul 08 21:20:17.574384 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.574339 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-v852d" podStartSLOduration=53.698234365 podStartE2EDuration="57.574324906s" totalImagesPullingTime="3.876090541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:17.572978657 +0000 UTC m=+275.483416077" watchObservedRunningTime="2026-07-08 21:20:17.574324906 +0000 UTC m=+275.484762322" Jul 08 21:20:17.577208 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.577176 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-wc7hd"] Jul 08 21:20:17.579735 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.579718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:17.582033 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.582013 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-j2rqn\"" Jul 08 21:20:17.582231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.582216 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 21:20:17.582303 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.582246 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 21:20:17.589775 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.589751 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-wc7hd"] Jul 08 21:20:17.677954 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.677919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rrth\" (UniqueName: \"kubernetes.io/projected/c04e0a32-8bd7-46cd-9435-0f74a7adb781-kube-api-access-6rrth\") pod \"downloads-685bfb5f64-wc7hd\" (UID: \"c04e0a32-8bd7-46cd-9435-0f74a7adb781\") " pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:17.778748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.778720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rrth\" (UniqueName: \"kubernetes.io/projected/c04e0a32-8bd7-46cd-9435-0f74a7adb781-kube-api-access-6rrth\") pod \"downloads-685bfb5f64-wc7hd\" (UID: \"c04e0a32-8bd7-46cd-9435-0f74a7adb781\") " pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:17.786766 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.786742 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rrth\" (UniqueName: \"kubernetes.io/projected/c04e0a32-8bd7-46cd-9435-0f74a7adb781-kube-api-access-6rrth\") pod \"downloads-685bfb5f64-wc7hd\" (UID: \"c04e0a32-8bd7-46cd-9435-0f74a7adb781\") " pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:17.889306 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:17.889276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:18.032253 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:18.032230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-wc7hd"] Jul 08 21:20:18.034603 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:18.034574 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04e0a32_8bd7_46cd_9435_0f74a7adb781.slice/crio-ac4a2941fe785fd6a260ff8d07fc853dbc41b8e122e72d65b59ccd889c30fea3 WatchSource:0}: Error finding container ac4a2941fe785fd6a260ff8d07fc853dbc41b8e122e72d65b59ccd889c30fea3: Status 404 returned error can't find the container with id ac4a2941fe785fd6a260ff8d07fc853dbc41b8e122e72d65b59ccd889c30fea3 Jul 08 21:20:18.550363 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:18.550325 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-wc7hd" event={"ID":"c04e0a32-8bd7-46cd-9435-0f74a7adb781","Type":"ContainerStarted","Data":"ac4a2941fe785fd6a260ff8d07fc853dbc41b8e122e72d65b59ccd889c30fea3"} Jul 08 21:20:23.134071 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:23.134029 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gfqn2" podUID="5d640c3d-da59-4e06-880a-e1b27b2f6013" Jul 08 21:20:23.134479 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:20:23.134098 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" Jul 08 21:20:23.574227 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:23.574156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:20:23.574452 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:23.574156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfqn2" Jul 08 21:20:26.859503 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.859469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:20:26.859503 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.859512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:20:26.862380 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.862325 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d640c3d-da59-4e06-880a-e1b27b2f6013-metrics-tls\") pod \"dns-default-gfqn2\" (UID: \"5d640c3d-da59-4e06-880a-e1b27b2f6013\") " pod="openshift-dns/dns-default-gfqn2" Jul 08 21:20:26.862380 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.862338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"image-registry-7f54f9458-4t7dl\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:20:26.878747 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.878724 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m6rnx\"" Jul 08 21:20:26.878747 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.878738 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t7n5m\"" Jul 08 21:20:26.886140 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.886120 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:20:26.886237 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:26.886222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfqn2" Jul 08 21:20:27.035034 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.035009 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gfqn2"] Jul 08 21:20:27.038024 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:27.037994 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d640c3d_da59_4e06_880a_e1b27b2f6013.slice/crio-9d1caf562744a37816009042af7b2ecd42808224d8c06d658ad1eb236013e061 WatchSource:0}: Error finding container 9d1caf562744a37816009042af7b2ecd42808224d8c06d658ad1eb236013e061: Status 404 returned error can't find the container with id 9d1caf562744a37816009042af7b2ecd42808224d8c06d658ad1eb236013e061 Jul 08 21:20:27.050814 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.050795 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f54f9458-4t7dl"] Jul 08 21:20:27.053476 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:20:27.053437 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f0522b_2a49_474a_9195_5cc1a9df98c0.slice/crio-c9574ecbdc1816ccd9129f390bdad78c058ed386c9b1c686121e677831d8cc87 WatchSource:0}: Error finding container c9574ecbdc1816ccd9129f390bdad78c058ed386c9b1c686121e677831d8cc87: Status 404 returned error can't find the container with id c9574ecbdc1816ccd9129f390bdad78c058ed386c9b1c686121e677831d8cc87 Jul 08 21:20:27.597688 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.597654 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" event={"ID":"76f0522b-2a49-474a-9195-5cc1a9df98c0","Type":"ContainerStarted","Data":"e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a"} Jul 08 21:20:27.597885 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.597709 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:20:27.597885 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.597725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" event={"ID":"76f0522b-2a49-474a-9195-5cc1a9df98c0","Type":"ContainerStarted","Data":"c9574ecbdc1816ccd9129f390bdad78c058ed386c9b1c686121e677831d8cc87"} Jul 08 21:20:27.598794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.598768 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gfqn2" event={"ID":"5d640c3d-da59-4e06-880a-e1b27b2f6013","Type":"ContainerStarted","Data":"9d1caf562744a37816009042af7b2ecd42808224d8c06d658ad1eb236013e061"} Jul 08 21:20:27.617456 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:27.617415 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" podStartSLOduration=287.617400551 podStartE2EDuration="4m47.617400551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:27.616399957 +0000 UTC m=+285.526837387" watchObservedRunningTime="2026-07-08 21:20:27.617400551 +0000 UTC m=+285.527837945" Jul 08 21:20:29.608843 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:29.608808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gfqn2" event={"ID":"5d640c3d-da59-4e06-880a-e1b27b2f6013","Type":"ContainerStarted","Data":"abb15f9fd3181c5b94c88744eb7f587792079cc20a8df4e59158848d8abd7a81"} Jul 08 21:20:30.620434 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:30.620392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gfqn2" event={"ID":"5d640c3d-da59-4e06-880a-e1b27b2f6013","Type":"ContainerStarted","Data":"e0ec9c9a636b3e2c5f1e10975c1d4add9b7f46e2e974c3b807b7d27a4e5c8da7"} Jul 08 21:20:30.621072 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:30.621049 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gfqn2" Jul 08 21:20:30.637466 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:30.636753 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gfqn2" podStartSLOduration=252.225562145 podStartE2EDuration="4m14.636735257s" totalImagesPullingTime="2.411173112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:30.636176196 +0000 UTC m=+288.546613637" watchObservedRunningTime="2026-07-08 21:20:30.636735257 +0000 UTC m=+288.547172675" Jul 08 21:20:39.669151 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.668828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-wc7hd" event={"ID":"c04e0a32-8bd7-46cd-9435-0f74a7adb781","Type":"ContainerStarted","Data":"ba5620362f54692e134d9d2f633e3f6d4d5df107bf1a17eaa5515550af26b5b4"} Jul 08 21:20:39.669151 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.668911 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:39.671108 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.671079 2578 generic.go:356] "Generic (PLEG): container finished" podID="6df65827-dae8-46cf-a3c3-dab3652ffe06" containerID="135e9b699a42a7480446d4a59c49b2d6cfb75f57182a422a1f665b46dbe47b54" exitCode=0 Jul 08 21:20:39.671236 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.671141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" event={"ID":"6df65827-dae8-46cf-a3c3-dab3652ffe06","Type":"ContainerDied","Data":"135e9b699a42a7480446d4a59c49b2d6cfb75f57182a422a1f665b46dbe47b54"} Jul 08 21:20:39.671449 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.671436 2578 scope.go:117] "RemoveContainer" containerID="135e9b699a42a7480446d4a59c49b2d6cfb75f57182a422a1f665b46dbe47b54" Jul 08 21:20:39.681324 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.681292 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-wc7hd" Jul 08 21:20:39.687814 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:39.687768 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-wc7hd" podStartSLOduration=1.98790796 podStartE2EDuration="22.687755283s" totalImagesPullingTime="20.699847323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:39.685483684 +0000 UTC m=+297.595921102" watchObservedRunningTime="2026-07-08 21:20:39.687755283 +0000 UTC m=+297.598192699" Jul 08 21:20:40.627577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:40.627546 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gfqn2" Jul 08 21:20:40.679700 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:40.679666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-t22lh" event={"ID":"6df65827-dae8-46cf-a3c3-dab3652ffe06","Type":"ContainerStarted","Data":"f35ccf415f3f89eca1b8c85023e7492da49d7ae5408f7a1e96e963edde838601"} Jul 08 21:20:42.489642 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:42.489613 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:20:42.492193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:42.492161 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:20:42.505310 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:42.505286 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 21:20:46.890895 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:46.890840 2578 patch_prober.go:28] interesting pod/image-registry-7f54f9458-4t7dl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 21:20:46.914134 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:46.890921 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 21:20:48.608099 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:48.608067 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:20:51.731639 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:51.731605 2578 generic.go:356] "Generic (PLEG): container finished" podID="006b5ab4-e339-4591-ba91-124fcf05b773" containerID="0564531107f75b7cac72ca63120edb75dddade1d059e78282f6f71311b257331" exitCode=0 Jul 08 21:20:51.732015 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:51.731652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" event={"ID":"006b5ab4-e339-4591-ba91-124fcf05b773","Type":"ContainerDied","Data":"0564531107f75b7cac72ca63120edb75dddade1d059e78282f6f71311b257331"} Jul 08 21:20:51.732015 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:51.731989 2578 scope.go:117] "RemoveContainer" containerID="0564531107f75b7cac72ca63120edb75dddade1d059e78282f6f71311b257331" Jul 08 21:20:52.737862 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:52.737828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-f85dw" event={"ID":"006b5ab4-e339-4591-ba91-124fcf05b773","Type":"ContainerStarted","Data":"2b7a4b7e6e04650e79d761b4f08e663e092694d4f2e486b927ac5f48766fd331"} Jul 08 21:20:53.180840 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:53.180807 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gfqn2_5d640c3d-da59-4e06-880a-e1b27b2f6013/dns/0.log" Jul 08 21:20:53.380478 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:53.380449 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gfqn2_5d640c3d-da59-4e06-880a-e1b27b2f6013/kube-rbac-proxy/0.log" Jul 08 21:20:54.379575 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:54.379546 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wlhfz_3c83b512-6ff0-402d-85a9-44883cb68f6a/dns-node-resolver/0.log" Jul 08 21:20:55.180491 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:55.180466 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wznnp_0235050f-9f13-48dd-93a1-d180b0fd9a21/serve-healthcheck-canary/0.log" Jul 08 21:20:55.751265 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:55.751172 2578 generic.go:356] "Generic (PLEG): container finished" podID="a990f07a-ea05-44ca-8843-571e01535039" containerID="d31dbc7bc09929083d69fcc5fa13fdfc7e6c94837f218656d8b8bcf74fcc4370" exitCode=0 Jul 08 21:20:55.751265 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:55.751252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" event={"ID":"a990f07a-ea05-44ca-8843-571e01535039","Type":"ContainerDied","Data":"d31dbc7bc09929083d69fcc5fa13fdfc7e6c94837f218656d8b8bcf74fcc4370"} Jul 08 21:20:55.751681 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:55.751603 2578 scope.go:117] "RemoveContainer" containerID="d31dbc7bc09929083d69fcc5fa13fdfc7e6c94837f218656d8b8bcf74fcc4370" Jul 08 21:20:56.757140 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:20:56.757102 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-42z8k" event={"ID":"a990f07a-ea05-44ca-8843-571e01535039","Type":"ContainerStarted","Data":"de24f4e4126334ddd54b50bb56b6e5eacabcebb0ce5aca0526a6303d952f55d0"} Jul 08 21:21:14.944061 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:14.944026 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f54f9458-4t7dl"] Jul 08 21:21:23.164696 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.164665 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:21:23.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.164962 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="alertmanager" containerID="cri-o://175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de" gracePeriod=120 Jul 08 21:21:23.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.164979 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="prom-label-proxy" containerID="cri-o://2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5" gracePeriod=120 Jul 08 21:21:23.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.165013 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-web" containerID="cri-o://a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a" gracePeriod=120 Jul 08 21:21:23.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.165014 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-metric" containerID="cri-o://f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84" gracePeriod=120 Jul 08 21:21:23.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.165043 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="config-reloader" containerID="cri-o://5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183" gracePeriod=120 Jul 08 21:21:23.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.164989 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy" containerID="cri-o://972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b" gracePeriod=120 Jul 08 21:21:23.878685 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878648 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5" exitCode=0 Jul 08 21:21:23.878685 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878675 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b" exitCode=0 Jul 08 21:21:23.878685 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878682 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183" exitCode=0 Jul 08 21:21:23.878685 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878692 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de" exitCode=0 Jul 08 21:21:23.878967 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5"} Jul 08 21:21:23.878967 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b"} Jul 08 21:21:23.878967 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183"} Jul 08 21:21:23.878967 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:23.878853 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de"} Jul 08 21:21:24.411760 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.411736 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:24.557165 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557083 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557165 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557120 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557165 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557158 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557178 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-config-volume\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557195 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-config-out\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557217 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-web\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557238 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557260 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg8lr\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-kube-api-access-cg8lr\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557296 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-main-db\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557318 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-tls-assets\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557355 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-web-config\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557380 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-cluster-tls-config\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557424 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557402 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-metrics-client-ca\") pod \"9cea011e-7f91-4866-87b9-71cde6307807\" (UID: \"9cea011e-7f91-4866-87b9-71cde6307807\") " Jul 08 21:21:24.557922 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557498 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:21:24.557922 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557822 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.558036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557941 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:21:24.558036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.557960 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:21:24.560372 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.560326 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.560733 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.560700 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.560937 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.560912 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:21:24.561232 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.561208 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.561527 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.561494 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-config-volume" (OuterVolumeSpecName: "config-volume") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.561608 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.561589 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-config-out" (OuterVolumeSpecName: "config-out") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:21:24.561772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.561752 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.562917 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.562894 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-kube-api-access-cg8lr" (OuterVolumeSpecName: "kube-api-access-cg8lr") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "kube-api-access-cg8lr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:21:24.565832 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.565810 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.572350 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.572330 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-web-config" (OuterVolumeSpecName: "web-config") pod "9cea011e-7f91-4866-87b9-71cde6307807" (UID: "9cea011e-7f91-4866-87b9-71cde6307807"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:24.659100 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659076 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659100 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659100 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-config-volume\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659110 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-config-out\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659119 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659127 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659136 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg8lr\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-kube-api-access-cg8lr\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659144 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9cea011e-7f91-4866-87b9-71cde6307807-alertmanager-main-db\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659152 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9cea011e-7f91-4866-87b9-71cde6307807-tls-assets\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659178 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-web-config\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659186 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-cluster-tls-config\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659194 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cea011e-7f91-4866-87b9-71cde6307807-metrics-client-ca\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.659231 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.659202 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9cea011e-7f91-4866-87b9-71cde6307807-secret-alertmanager-main-tls\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:24.886137 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886111 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84" exitCode=0 Jul 08 21:21:24.886137 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886134 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cea011e-7f91-4866-87b9-71cde6307807" containerID="a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a" exitCode=0 Jul 08 21:21:24.886315 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886198 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:24.886315 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84"} Jul 08 21:21:24.886385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a"} Jul 08 21:21:24.886385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9cea011e-7f91-4866-87b9-71cde6307807","Type":"ContainerDied","Data":"f35a1a70f1e59715c8869e08d862ae63243f72bf13136fed9a4b2ebc90a137e5"} Jul 08 21:21:24.886442 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.886386 2578 scope.go:117] "RemoveContainer" containerID="2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5" Jul 08 21:21:24.894239 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.894225 2578 scope.go:117] "RemoveContainer" containerID="f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84" Jul 08 21:21:24.901049 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.901034 2578 scope.go:117] "RemoveContainer" containerID="972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b" Jul 08 21:21:24.906747 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.906727 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:21:24.908972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.908916 2578 scope.go:117] "RemoveContainer" containerID="a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a" Jul 08 21:21:24.910649 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.910628 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:21:24.915701 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.915686 2578 scope.go:117] "RemoveContainer" containerID="5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183" Jul 08 21:21:24.922504 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.922489 2578 scope.go:117] "RemoveContainer" containerID="175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de" Jul 08 21:21:24.931074 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.930788 2578 scope.go:117] "RemoveContainer" containerID="19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3" Jul 08 21:21:24.938207 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938191 2578 scope.go:117] "RemoveContainer" containerID="2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5" Jul 08 21:21:24.938445 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.938427 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5\": container with ID starting with 2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5 not found: ID does not exist" containerID="2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5" Jul 08 21:21:24.938493 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938453 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5"} err="failed to get container status \"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5\": rpc error: code = NotFound desc = could not find container \"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5\": container with ID starting with 2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5 not found: ID does not exist" Jul 08 21:21:24.938493 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938484 2578 scope.go:117] "RemoveContainer" containerID="f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84" Jul 08 21:21:24.938690 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.938673 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84\": container with ID starting with f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84 not found: ID does not exist" containerID="f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84" Jul 08 21:21:24.938744 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938695 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84"} err="failed to get container status \"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84\": rpc error: code = NotFound desc = could not find container \"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84\": container with ID starting with f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84 not found: ID does not exist" Jul 08 21:21:24.938744 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938711 2578 scope.go:117] "RemoveContainer" containerID="972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b" Jul 08 21:21:24.938939 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.938921 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b\": container with ID starting with 972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b not found: ID does not exist" containerID="972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b" Jul 08 21:21:24.938996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938947 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b"} err="failed to get container status \"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b\": rpc error: code = NotFound desc = could not find container \"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b\": container with ID starting with 972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b not found: ID does not exist" Jul 08 21:21:24.938996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.938963 2578 scope.go:117] "RemoveContainer" containerID="a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a" Jul 08 21:21:24.939190 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.939174 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a\": container with ID starting with a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a not found: ID does not exist" containerID="a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a" Jul 08 21:21:24.939237 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939194 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a"} err="failed to get container status \"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a\": rpc error: code = NotFound desc = could not find container \"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a\": container with ID starting with a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a not found: ID does not exist" Jul 08 21:21:24.939237 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939207 2578 scope.go:117] "RemoveContainer" containerID="5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183" Jul 08 21:21:24.939427 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.939411 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183\": container with ID starting with 5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183 not found: ID does not exist" containerID="5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183" Jul 08 21:21:24.939470 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939431 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183"} err="failed to get container status \"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183\": rpc error: code = NotFound desc = could not find container \"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183\": container with ID starting with 5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183 not found: ID does not exist" Jul 08 21:21:24.939470 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939444 2578 scope.go:117] "RemoveContainer" containerID="175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de" Jul 08 21:21:24.939660 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.939644 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de\": container with ID starting with 175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de not found: ID does not exist" containerID="175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de" Jul 08 21:21:24.939712 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939666 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de"} err="failed to get container status \"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de\": rpc error: code = NotFound desc = could not find container \"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de\": container with ID starting with 175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de not found: ID does not exist" Jul 08 21:21:24.939712 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939679 2578 scope.go:117] "RemoveContainer" containerID="19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3" Jul 08 21:21:24.939920 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:24.939900 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3\": container with ID starting with 19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3 not found: ID does not exist" containerID="19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3" Jul 08 21:21:24.939972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939923 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3"} err="failed to get container status \"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3\": rpc error: code = NotFound desc = could not find container \"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3\": container with ID starting with 19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3 not found: ID does not exist" Jul 08 21:21:24.939972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.939936 2578 scope.go:117] "RemoveContainer" containerID="2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5" Jul 08 21:21:24.940146 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940129 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5"} err="failed to get container status \"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5\": rpc error: code = NotFound desc = could not find container \"2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5\": container with ID starting with 2eeecf3312b09b11568ffa18f99038a0be366a6f7c6a3653fdfecdc268b1dcd5 not found: ID does not exist" Jul 08 21:21:24.940189 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940147 2578 scope.go:117] "RemoveContainer" containerID="f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84" Jul 08 21:21:24.940332 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940314 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84"} err="failed to get container status \"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84\": rpc error: code = NotFound desc = could not find container \"f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84\": container with ID starting with f68174a106e246ec7f0726b0b33a390c59361fd141e9d940308a43898f4d7d84 not found: ID does not exist" Jul 08 21:21:24.940376 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940334 2578 scope.go:117] "RemoveContainer" containerID="972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b" Jul 08 21:21:24.940531 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940515 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b"} err="failed to get container status \"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b\": rpc error: code = NotFound desc = could not find container \"972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b\": container with ID starting with 972708d3df3c042a31751b31353d72826d8329a392dc0c396145c5612ef58a4b not found: ID does not exist" Jul 08 21:21:24.940585 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940531 2578 scope.go:117] "RemoveContainer" containerID="a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a" Jul 08 21:21:24.940716 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940701 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a"} err="failed to get container status \"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a\": rpc error: code = NotFound desc = could not find container \"a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a\": container with ID starting with a5b437812e06dc14e6ed584d32b2d9a9ac58a680ce6e1c62e509400cc15f913a not found: ID does not exist" Jul 08 21:21:24.940764 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940716 2578 scope.go:117] "RemoveContainer" containerID="5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183" Jul 08 21:21:24.940889 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940855 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183"} err="failed to get container status \"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183\": rpc error: code = NotFound desc = could not find container \"5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183\": container with ID starting with 5a52dbcca5d1d7003d096aa715a95f7489e8b238e199dc1e9b7feaa27ee25183 not found: ID does not exist" Jul 08 21:21:24.940926 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.940890 2578 scope.go:117] "RemoveContainer" containerID="175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de" Jul 08 21:21:24.941074 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.941059 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de"} err="failed to get container status \"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de\": rpc error: code = NotFound desc = could not find container \"175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de\": container with ID starting with 175f14e38e37a0be4d2db8c996c4b20a0a6fdc15ebf519b0c5ddba78d2dbc1de not found: ID does not exist" Jul 08 21:21:24.941121 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.941074 2578 scope.go:117] "RemoveContainer" containerID="19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3" Jul 08 21:21:24.941249 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.941235 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3"} err="failed to get container status \"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3\": rpc error: code = NotFound desc = could not find container \"19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3\": container with ID starting with 19ff2b0c91fa825dfb261f3d811af9351e16fc38ad74da81da62c6f3be57bfa3 not found: ID does not exist" Jul 08 21:21:24.947550 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947529 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:21:24.947847 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947833 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-web" Jul 08 21:21:24.947910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947850 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-web" Jul 08 21:21:24.947910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947859 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-metric" Jul 08 21:21:24.947910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947882 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-metric" Jul 08 21:21:24.947910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947894 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy" Jul 08 21:21:24.947910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947901 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy" Jul 08 21:21:24.947910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947911 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="config-reloader" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947917 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="config-reloader" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947952 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="init-config-reloader" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947958 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="init-config-reloader" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947981 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="alertmanager" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947987 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="alertmanager" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947994 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="prom-label-proxy" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.947999 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="prom-label-proxy" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.948053 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.948062 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-metric" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.948075 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="kube-rbac-proxy-web" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.948081 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="alertmanager" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.948087 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="config-reloader" Jul 08 21:21:24.948095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.948094 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cea011e-7f91-4866-87b9-71cde6307807" containerName="prom-label-proxy" Jul 08 21:21:24.952122 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.952104 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:24.954767 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.954745 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 21:21:24.954857 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.954829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 21:21:24.955132 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955109 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 21:21:24.955276 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955258 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kf6ln\"" Jul 08 21:21:24.955382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955326 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 21:21:24.955382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955332 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 21:21:24.955382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955366 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 21:21:24.955540 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955325 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 21:21:24.955540 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.955332 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 21:21:24.959971 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.959953 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 21:21:24.967329 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:24.967306 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:21:25.062862 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.062832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/086c9898-c0e9-4ef1-8379-61b44ae0a57d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063058 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.062898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063058 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.062916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063058 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.062935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74mfx\" (UniqueName: \"kubernetes.io/projected/086c9898-c0e9-4ef1-8379-61b44ae0a57d-kube-api-access-74mfx\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063058 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.062953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-web-config\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063058 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/086c9898-c0e9-4ef1-8379-61b44ae0a57d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063075 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/086c9898-c0e9-4ef1-8379-61b44ae0a57d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/086c9898-c0e9-4ef1-8379-61b44ae0a57d-config-out\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-config-volume\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.063340 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.063295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/086c9898-c0e9-4ef1-8379-61b44ae0a57d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164604 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/086c9898-c0e9-4ef1-8379-61b44ae0a57d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164604 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/086c9898-c0e9-4ef1-8379-61b44ae0a57d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164604 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/086c9898-c0e9-4ef1-8379-61b44ae0a57d-config-out\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164841 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164619 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164841 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-config-volume\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164841 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.164841 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.165076 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/086c9898-c0e9-4ef1-8379-61b44ae0a57d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.165132 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.164721 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/086c9898-c0e9-4ef1-8379-61b44ae0a57d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.165183 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.165150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/086c9898-c0e9-4ef1-8379-61b44ae0a57d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.165233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.165218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.165285 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.165249 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.165285 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.165280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74mfx\" (UniqueName: \"kubernetes.io/projected/086c9898-c0e9-4ef1-8379-61b44ae0a57d-kube-api-access-74mfx\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.167556 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.167525 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/086c9898-c0e9-4ef1-8379-61b44ae0a57d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.167651 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.165309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-web-config\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.168896 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.168014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.168896 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.168064 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.168896 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.168073 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/086c9898-c0e9-4ef1-8379-61b44ae0a57d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.168896 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.168227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/086c9898-c0e9-4ef1-8379-61b44ae0a57d-config-out\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.168896 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.168491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.168896 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.168595 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-config-volume\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.169245 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.169103 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/086c9898-c0e9-4ef1-8379-61b44ae0a57d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.173243 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.170041 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.173243 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.170206 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-web-config\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.173938 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.173919 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/086c9898-c0e9-4ef1-8379-61b44ae0a57d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.175923 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.175907 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74mfx\" (UniqueName: \"kubernetes.io/projected/086c9898-c0e9-4ef1-8379-61b44ae0a57d-kube-api-access-74mfx\") pod \"alertmanager-main-0\" (UID: \"086c9898-c0e9-4ef1-8379-61b44ae0a57d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.261635 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.261614 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:21:25.393014 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.392987 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:21:25.395447 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:21:25.395423 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086c9898_c0e9_4ef1_8379_61b44ae0a57d.slice/crio-abf074a41ecbb8ca3f5693c9722a80ebe5782f505fe35e9a2cdb7232e7614d6a WatchSource:0}: Error finding container abf074a41ecbb8ca3f5693c9722a80ebe5782f505fe35e9a2cdb7232e7614d6a: Status 404 returned error can't find the container with id abf074a41ecbb8ca3f5693c9722a80ebe5782f505fe35e9a2cdb7232e7614d6a Jul 08 21:21:25.892183 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.892147 2578 generic.go:356] "Generic (PLEG): container finished" podID="086c9898-c0e9-4ef1-8379-61b44ae0a57d" containerID="74dfbbc707abd1ea43132a13e2424ad2244c5d880dbb9e1d4d1cc1d0b7f65401" exitCode=0 Jul 08 21:21:25.892533 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.892230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerDied","Data":"74dfbbc707abd1ea43132a13e2424ad2244c5d880dbb9e1d4d1cc1d0b7f65401"} Jul 08 21:21:25.892533 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:25.892295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"abf074a41ecbb8ca3f5693c9722a80ebe5782f505fe35e9a2cdb7232e7614d6a"} Jul 08 21:21:26.615177 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.615102 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cea011e-7f91-4866-87b9-71cde6307807" path="/var/lib/kubelet/pods/9cea011e-7f91-4866-87b9-71cde6307807/volumes" Jul 08 21:21:26.900800 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.900765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"a0b38bf690151c498ecce41a60568549a49048e3404dfc9450351c2bf40e67c0"} Jul 08 21:21:26.901139 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.900833 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"35746a8df7575776bf12569f81788099bdb95a30562869af6c49e7d1907fe369"} Jul 08 21:21:26.901139 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.900883 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"3e62c457cf8bcf8663e2c0b9a50963571f1c3f433ed3dcd0fad20febadb50121"} Jul 08 21:21:26.901139 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.900918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"2c2a5019ee8229d48d0f809560e89353ba775cb9949a9ff5a8de014439e6958e"} Jul 08 21:21:26.901139 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.900943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"77223c885d177c77540469f8fc060c46cbce454c31b2e957fc8422a4b5a24a4c"} Jul 08 21:21:26.901139 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.900966 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"086c9898-c0e9-4ef1-8379-61b44ae0a57d","Type":"ContainerStarted","Data":"a0755b2277f380a1b4be6ed8324b2d2cdfde331bb6063371c1da3ebe7db2737f"} Jul 08 21:21:26.926679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:26.926636 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.909270251 podStartE2EDuration="2.92661978s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.349529ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:21:26.924675763 +0000 UTC m=+344.835113198" watchObservedRunningTime="2026-07-08 21:21:26.92661978 +0000 UTC m=+344.837057197" Jul 08 21:21:27.191009 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.190927 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-67cfbd585c-6277f"] Jul 08 21:21:27.194382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.194355 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.197301 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.197272 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 21:21:27.197430 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.197418 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pzthq\"" Jul 08 21:21:27.197506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.197417 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 21:21:27.197593 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.197576 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 21:21:27.197728 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.197701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 21:21:27.197852 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.197827 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 21:21:27.203400 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.203381 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 21:21:27.212415 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.212387 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-67cfbd585c-6277f"] Jul 08 21:21:27.286285 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-federate-client-tls\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286458 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-metrics-client-ca\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286458 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-secret-telemeter-client\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286458 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286392 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286458 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286434 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-telemeter-trusted-ca-bundle\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286692 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-serving-certs-ca-bundle\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286692 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-telemeter-client-tls\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.286692 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.286544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhvk4\" (UniqueName: \"kubernetes.io/projected/0e605723-b6fb-4323-a009-2ad2e3bbb739-kube-api-access-jhvk4\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.387707 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-metrics-client-ca\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.387906 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-secret-telemeter-client\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.387906 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.387906 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-telemeter-trusted-ca-bundle\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.388080 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-serving-certs-ca-bundle\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.388080 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387934 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-telemeter-client-tls\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.388080 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhvk4\" (UniqueName: \"kubernetes.io/projected/0e605723-b6fb-4323-a009-2ad2e3bbb739-kube-api-access-jhvk4\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.388080 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.387983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-federate-client-tls\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.389899 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.389739 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-telemeter-trusted-ca-bundle\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.390581 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.390550 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-metrics-client-ca\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.391121 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.391096 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e605723-b6fb-4323-a009-2ad2e3bbb739-serving-certs-ca-bundle\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.393073 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.393049 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-federate-client-tls\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.394899 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.394851 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.394991 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.394967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-telemeter-client-tls\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.397856 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.397831 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0e605723-b6fb-4323-a009-2ad2e3bbb739-secret-telemeter-client\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.399289 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.399261 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhvk4\" (UniqueName: \"kubernetes.io/projected/0e605723-b6fb-4323-a009-2ad2e3bbb739-kube-api-access-jhvk4\") pod \"telemeter-client-67cfbd585c-6277f\" (UID: \"0e605723-b6fb-4323-a009-2ad2e3bbb739\") " pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.506653 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.506576 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" Jul 08 21:21:27.649662 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.649631 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-67cfbd585c-6277f"] Jul 08 21:21:27.653520 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:21:27.653495 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e605723_b6fb_4323_a009_2ad2e3bbb739.slice/crio-e6bcb221021da1fa15f5052914dd9c881cbdf6ae65c9ae51e7f4a53d1c308a5c WatchSource:0}: Error finding container e6bcb221021da1fa15f5052914dd9c881cbdf6ae65c9ae51e7f4a53d1c308a5c: Status 404 returned error can't find the container with id e6bcb221021da1fa15f5052914dd9c881cbdf6ae65c9ae51e7f4a53d1c308a5c Jul 08 21:21:27.656214 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.656194 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:21:27.906779 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:27.906744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" event={"ID":"0e605723-b6fb-4323-a009-2ad2e3bbb739","Type":"ContainerStarted","Data":"e6bcb221021da1fa15f5052914dd9c881cbdf6ae65c9ae51e7f4a53d1c308a5c"} Jul 08 21:21:30.921720 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:30.921686 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" event={"ID":"0e605723-b6fb-4323-a009-2ad2e3bbb739","Type":"ContainerStarted","Data":"495e287326e8ee4912f5b3e148ce6f5f5dffd9e554037ffbaec3c9d564cba1a9"} Jul 08 21:21:30.922116 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:30.921741 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" event={"ID":"0e605723-b6fb-4323-a009-2ad2e3bbb739","Type":"ContainerStarted","Data":"1f100af50ec48208b4d71f4e53b8b53b517c830bb76fe8ba80eb308004935bb9"} Jul 08 21:21:30.922116 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:30.921759 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" event={"ID":"0e605723-b6fb-4323-a009-2ad2e3bbb739","Type":"ContainerStarted","Data":"c0c693ee11aeeb985d83abdd5244d8c5aa34e8c392560b9118a1b5aecca5225c"} Jul 08 21:21:30.944180 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:30.944132 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-67cfbd585c-6277f" podStartSLOduration=1.383840627 podStartE2EDuration="3.94411485s" totalImagesPullingTime="2.560274223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:21:30.942193672 +0000 UTC m=+348.852631089" watchObservedRunningTime="2026-07-08 21:21:30.94411485 +0000 UTC m=+348.854552270" Jul 08 21:21:39.964710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:39.964603 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" containerName="registry" containerID="cri-o://e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a" gracePeriod=30 Jul 08 21:21:40.214564 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.214539 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:21:40.296571 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296494 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-bound-sa-token\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296571 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296528 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxnl4\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-kube-api-access-rxnl4\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296571 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296551 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-trusted-ca\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296580 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76f0522b-2a49-474a-9195-5cc1a9df98c0-ca-trust-extracted\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296604 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-image-registry-private-configuration\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296628 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-certificates\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296654 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-installation-pull-secrets\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.296821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.296705 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") pod \"76f0522b-2a49-474a-9195-5cc1a9df98c0\" (UID: \"76f0522b-2a49-474a-9195-5cc1a9df98c0\") " Jul 08 21:21:40.297261 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.297222 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:21:40.297435 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.297410 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 21:21:40.299396 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.299346 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:40.299590 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.299565 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:21:40.299655 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.299625 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-kube-api-access-rxnl4" (OuterVolumeSpecName: "kube-api-access-rxnl4") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "kube-api-access-rxnl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:21:40.299718 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.299691 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:21:40.299718 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.299700 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:21:40.305136 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.305112 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76f0522b-2a49-474a-9195-5cc1a9df98c0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "76f0522b-2a49-474a-9195-5cc1a9df98c0" (UID: "76f0522b-2a49-474a-9195-5cc1a9df98c0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:21:40.398078 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398055 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-bound-sa-token\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398078 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398075 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxnl4\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-kube-api-access-rxnl4\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398085 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-trusted-ca\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398093 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76f0522b-2a49-474a-9195-5cc1a9df98c0-ca-trust-extracted\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398103 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-image-registry-private-configuration\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398112 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-certificates\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398120 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76f0522b-2a49-474a-9195-5cc1a9df98c0-installation-pull-secrets\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.398230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.398128 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76f0522b-2a49-474a-9195-5cc1a9df98c0-registry-tls\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:21:40.964174 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.964140 2578 generic.go:356] "Generic (PLEG): container finished" podID="76f0522b-2a49-474a-9195-5cc1a9df98c0" containerID="e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a" exitCode=0 Jul 08 21:21:40.964341 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.964188 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" Jul 08 21:21:40.964341 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.964223 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" event={"ID":"76f0522b-2a49-474a-9195-5cc1a9df98c0","Type":"ContainerDied","Data":"e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a"} Jul 08 21:21:40.964341 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.964274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f54f9458-4t7dl" event={"ID":"76f0522b-2a49-474a-9195-5cc1a9df98c0","Type":"ContainerDied","Data":"c9574ecbdc1816ccd9129f390bdad78c058ed386c9b1c686121e677831d8cc87"} Jul 08 21:21:40.964341 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.964293 2578 scope.go:117] "RemoveContainer" containerID="e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a" Jul 08 21:21:40.972503 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.972487 2578 scope.go:117] "RemoveContainer" containerID="e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a" Jul 08 21:21:40.972786 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:21:40.972761 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a\": container with ID starting with e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a not found: ID does not exist" containerID="e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a" Jul 08 21:21:40.972832 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.972783 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a"} err="failed to get container status \"e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a\": rpc error: code = NotFound desc = could not find container \"e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a\": container with ID starting with e89e048bf9244fab4d67abe76da4f2daa2bc4796f0c2d50839cea4428e98fd9a not found: ID does not exist" Jul 08 21:21:40.984337 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.984315 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f54f9458-4t7dl"] Jul 08 21:21:40.986983 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:40.986966 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f54f9458-4t7dl"] Jul 08 21:21:42.615803 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:21:42.615773 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" path="/var/lib/kubelet/pods/76f0522b-2a49-474a-9195-5cc1a9df98c0/volumes" Jul 08 21:23:51.776111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.776077 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw"] Jul 08 21:23:51.776535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.776390 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" containerName="registry" Jul 08 21:23:51.776535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.776403 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" containerName="registry" Jul 08 21:23:51.776535 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.776480 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f0522b-2a49-474a-9195-5cc1a9df98c0" containerName="registry" Jul 08 21:23:51.779513 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.779497 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:51.781775 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.781756 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 21:23:51.782732 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.782717 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 21:23:51.782821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.782805 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-crzgl\"" Jul 08 21:23:51.786823 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.786802 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw"] Jul 08 21:23:51.923433 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.923406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf5lx\" (UniqueName: \"kubernetes.io/projected/db9a962c-ae3a-4432-8c3f-46254b304ecd-kube-api-access-mf5lx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:51.923574 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.923440 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:51.923574 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:51.923507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.023949 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.023913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.024136 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.023977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5lx\" (UniqueName: \"kubernetes.io/projected/db9a962c-ae3a-4432-8c3f-46254b304ecd-kube-api-access-mf5lx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.024136 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.024005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.024375 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.024350 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.024375 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.024366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.032161 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.032101 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf5lx\" (UniqueName: \"kubernetes.io/projected/db9a962c-ae3a-4432-8c3f-46254b304ecd-kube-api-access-mf5lx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.089646 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.089621 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:23:52.216105 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.216083 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw"] Jul 08 21:23:52.218369 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:23:52.218344 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9a962c_ae3a_4432_8c3f_46254b304ecd.slice/crio-7f66360e4f068369b5cdc72856060761e4cef8b055081a28fd0a909b05ddc9cd WatchSource:0}: Error finding container 7f66360e4f068369b5cdc72856060761e4cef8b055081a28fd0a909b05ddc9cd: Status 404 returned error can't find the container with id 7f66360e4f068369b5cdc72856060761e4cef8b055081a28fd0a909b05ddc9cd Jul 08 21:23:52.499572 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:23:52.499534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" event={"ID":"db9a962c-ae3a-4432-8c3f-46254b304ecd","Type":"ContainerStarted","Data":"7f66360e4f068369b5cdc72856060761e4cef8b055081a28fd0a909b05ddc9cd"} Jul 08 21:24:01.538415 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:01.538383 2578 generic.go:356] "Generic (PLEG): container finished" podID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerID="cff7e96b9a67af31cd3cd98e06a43befdb4939aab9875e68aaa128e2975c6af3" exitCode=0 Jul 08 21:24:01.538415 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:01.538419 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" event={"ID":"db9a962c-ae3a-4432-8c3f-46254b304ecd","Type":"ContainerDied","Data":"cff7e96b9a67af31cd3cd98e06a43befdb4939aab9875e68aaa128e2975c6af3"} Jul 08 21:24:05.556575 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:05.556543 2578 generic.go:356] "Generic (PLEG): container finished" podID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerID="cc1579237b4511485a50a3598ab5ad7878185958c58d6c7f715ce87c0c97834f" exitCode=0 Jul 08 21:24:05.556946 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:05.556624 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" event={"ID":"db9a962c-ae3a-4432-8c3f-46254b304ecd","Type":"ContainerDied","Data":"cc1579237b4511485a50a3598ab5ad7878185958c58d6c7f715ce87c0c97834f"} Jul 08 21:24:13.597748 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:13.597670 2578 generic.go:356] "Generic (PLEG): container finished" podID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerID="bc3550d7507f046a78a1f49529eba90e0e18febe28fcf1d056f4b38a6da8d0b1" exitCode=0 Jul 08 21:24:13.598104 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:13.597765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" event={"ID":"db9a962c-ae3a-4432-8c3f-46254b304ecd","Type":"ContainerDied","Data":"bc3550d7507f046a78a1f49529eba90e0e18febe28fcf1d056f4b38a6da8d0b1"} Jul 08 21:24:14.772480 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.772454 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:24:14.819941 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.819914 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf5lx\" (UniqueName: \"kubernetes.io/projected/db9a962c-ae3a-4432-8c3f-46254b304ecd-kube-api-access-mf5lx\") pod \"db9a962c-ae3a-4432-8c3f-46254b304ecd\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " Jul 08 21:24:14.820071 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.819999 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-bundle\") pod \"db9a962c-ae3a-4432-8c3f-46254b304ecd\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " Jul 08 21:24:14.820071 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.820043 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-util\") pod \"db9a962c-ae3a-4432-8c3f-46254b304ecd\" (UID: \"db9a962c-ae3a-4432-8c3f-46254b304ecd\") " Jul 08 21:24:14.820623 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.820600 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-bundle" (OuterVolumeSpecName: "bundle") pod "db9a962c-ae3a-4432-8c3f-46254b304ecd" (UID: "db9a962c-ae3a-4432-8c3f-46254b304ecd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:24:14.822200 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.822176 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9a962c-ae3a-4432-8c3f-46254b304ecd-kube-api-access-mf5lx" (OuterVolumeSpecName: "kube-api-access-mf5lx") pod "db9a962c-ae3a-4432-8c3f-46254b304ecd" (UID: "db9a962c-ae3a-4432-8c3f-46254b304ecd"). InnerVolumeSpecName "kube-api-access-mf5lx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:24:14.824272 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.824248 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-util" (OuterVolumeSpecName: "util") pod "db9a962c-ae3a-4432-8c3f-46254b304ecd" (UID: "db9a962c-ae3a-4432-8c3f-46254b304ecd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:24:14.920800 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.920776 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:14.920970 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.920800 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf5lx\" (UniqueName: \"kubernetes.io/projected/db9a962c-ae3a-4432-8c3f-46254b304ecd-kube-api-access-mf5lx\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:14.920970 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:14.920816 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db9a962c-ae3a-4432-8c3f-46254b304ecd-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:15.607975 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:15.607943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" event={"ID":"db9a962c-ae3a-4432-8c3f-46254b304ecd","Type":"ContainerDied","Data":"7f66360e4f068369b5cdc72856060761e4cef8b055081a28fd0a909b05ddc9cd"} Jul 08 21:24:15.608145 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:15.607991 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f66360e4f068369b5cdc72856060761e4cef8b055081a28fd0a909b05ddc9cd" Jul 08 21:24:15.608145 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:15.607951 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n9vfw" Jul 08 21:24:19.865736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.865708 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5"] Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866052 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="util" Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866064 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="util" Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866081 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="pull" Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866086 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="pull" Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866104 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="extract" Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866109 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="extract" Jul 08 21:24:19.866185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.866164 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db9a962c-ae3a-4432-8c3f-46254b304ecd" containerName="extract" Jul 08 21:24:19.868290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.868275 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:19.871084 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.871061 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 21:24:19.871084 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.871077 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-b4rsk\"" Jul 08 21:24:19.871269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.871078 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:24:19.881977 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.881958 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5"] Jul 08 21:24:19.959065 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.959038 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eeba52d6-9483-4ae5-9ec2-76d0980fe377-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hqzh5\" (UID: \"eeba52d6-9483-4ae5-9ec2-76d0980fe377\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:19.959196 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:19.959083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksf72\" (UniqueName: \"kubernetes.io/projected/eeba52d6-9483-4ae5-9ec2-76d0980fe377-kube-api-access-ksf72\") pod \"cert-manager-operator-controller-manager-55c444bf86-hqzh5\" (UID: \"eeba52d6-9483-4ae5-9ec2-76d0980fe377\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:20.059815 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.059780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eeba52d6-9483-4ae5-9ec2-76d0980fe377-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hqzh5\" (UID: \"eeba52d6-9483-4ae5-9ec2-76d0980fe377\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:20.059985 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.059843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksf72\" (UniqueName: \"kubernetes.io/projected/eeba52d6-9483-4ae5-9ec2-76d0980fe377-kube-api-access-ksf72\") pod \"cert-manager-operator-controller-manager-55c444bf86-hqzh5\" (UID: \"eeba52d6-9483-4ae5-9ec2-76d0980fe377\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:20.060179 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.060162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eeba52d6-9483-4ae5-9ec2-76d0980fe377-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hqzh5\" (UID: \"eeba52d6-9483-4ae5-9ec2-76d0980fe377\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:20.068346 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.068326 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksf72\" (UniqueName: \"kubernetes.io/projected/eeba52d6-9483-4ae5-9ec2-76d0980fe377-kube-api-access-ksf72\") pod \"cert-manager-operator-controller-manager-55c444bf86-hqzh5\" (UID: \"eeba52d6-9483-4ae5-9ec2-76d0980fe377\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:20.177045 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.177017 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" Jul 08 21:24:20.308574 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.308537 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5"] Jul 08 21:24:20.311466 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:20.311442 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeba52d6_9483_4ae5_9ec2_76d0980fe377.slice/crio-a15e46fa62f66059821715e0ad737c0e4721109c14eab85bcfd60754b847ba58 WatchSource:0}: Error finding container a15e46fa62f66059821715e0ad737c0e4721109c14eab85bcfd60754b847ba58: Status 404 returned error can't find the container with id a15e46fa62f66059821715e0ad737c0e4721109c14eab85bcfd60754b847ba58 Jul 08 21:24:20.631168 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:20.631136 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" event={"ID":"eeba52d6-9483-4ae5-9ec2-76d0980fe377","Type":"ContainerStarted","Data":"a15e46fa62f66059821715e0ad737c0e4721109c14eab85bcfd60754b847ba58"} Jul 08 21:24:22.642290 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:22.642258 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" event={"ID":"eeba52d6-9483-4ae5-9ec2-76d0980fe377","Type":"ContainerStarted","Data":"05b75833d9ca267c204de4a7b844d65eeece05600ef9bd63065f0370f1e0a5bb"} Jul 08 21:24:22.664414 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:22.664354 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hqzh5" podStartSLOduration=1.9608876629999998 podStartE2EDuration="3.664339913s" totalImagesPullingTime="1.70345225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:24:22.66161774 +0000 UTC m=+520.572055157" watchObservedRunningTime="2026-07-08 21:24:22.664339913 +0000 UTC m=+520.574777330" Jul 08 21:24:23.962311 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.962282 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4"] Jul 08 21:24:23.964797 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.964780 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:23.967227 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.967195 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-crzgl\"" Jul 08 21:24:23.967343 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.967196 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 21:24:23.968374 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.968358 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 21:24:23.980111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.980087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4"] Jul 08 21:24:23.992562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.992531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cndhk\" (UniqueName: \"kubernetes.io/projected/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-kube-api-access-cndhk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:23.992751 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.992732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:23.992931 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:23.992915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.093639 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.093606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cndhk\" (UniqueName: \"kubernetes.io/projected/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-kube-api-access-cndhk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.093801 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.093644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.093801 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.093669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.094028 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.094009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.094107 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.094084 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.103088 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.103069 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cndhk\" (UniqueName: \"kubernetes.io/projected/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-kube-api-access-cndhk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.273797 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.273724 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:24.408145 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.408122 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4"] Jul 08 21:24:24.410263 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:24.410233 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5970bc5a_0097_4d15_b3a3_0c4ce9a68329.slice/crio-c3e7c8daf4c583ca35573bb368968dfc9d43f60d1ec2e3ef029a3188fa81bae9 WatchSource:0}: Error finding container c3e7c8daf4c583ca35573bb368968dfc9d43f60d1ec2e3ef029a3188fa81bae9: Status 404 returned error can't find the container with id c3e7c8daf4c583ca35573bb368968dfc9d43f60d1ec2e3ef029a3188fa81bae9 Jul 08 21:24:24.652404 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.652376 2578 generic.go:356] "Generic (PLEG): container finished" podID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerID="6af3d80f58091d4fc380e44977770e564a3216aeb81fc1eb0a2e066a0f868c2c" exitCode=0 Jul 08 21:24:24.652591 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.652414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" event={"ID":"5970bc5a-0097-4d15-b3a3-0c4ce9a68329","Type":"ContainerDied","Data":"6af3d80f58091d4fc380e44977770e564a3216aeb81fc1eb0a2e066a0f868c2c"} Jul 08 21:24:24.652591 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:24.652454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" event={"ID":"5970bc5a-0097-4d15-b3a3-0c4ce9a68329","Type":"ContainerStarted","Data":"c3e7c8daf4c583ca35573bb368968dfc9d43f60d1ec2e3ef029a3188fa81bae9"} Jul 08 21:24:26.141055 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.141015 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zh28z"] Jul 08 21:24:26.147952 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.147930 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.150754 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.150728 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 21:24:26.150919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.150753 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 21:24:26.150919 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.150727 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wtc44\"" Jul 08 21:24:26.151813 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.151792 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zh28z"] Jul 08 21:24:26.213316 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.213286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzpp2\" (UniqueName: \"kubernetes.io/projected/122e2c56-031f-4af3-af5c-a05f0602148c-kube-api-access-wzpp2\") pod \"cert-manager-cainjector-559cdd8c87-zh28z\" (UID: \"122e2c56-031f-4af3-af5c-a05f0602148c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.213474 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.213333 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/122e2c56-031f-4af3-af5c-a05f0602148c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zh28z\" (UID: \"122e2c56-031f-4af3-af5c-a05f0602148c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.314292 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.314259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzpp2\" (UniqueName: \"kubernetes.io/projected/122e2c56-031f-4af3-af5c-a05f0602148c-kube-api-access-wzpp2\") pod \"cert-manager-cainjector-559cdd8c87-zh28z\" (UID: \"122e2c56-031f-4af3-af5c-a05f0602148c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.314465 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.314336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/122e2c56-031f-4af3-af5c-a05f0602148c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zh28z\" (UID: \"122e2c56-031f-4af3-af5c-a05f0602148c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.322179 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.322152 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/122e2c56-031f-4af3-af5c-a05f0602148c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zh28z\" (UID: \"122e2c56-031f-4af3-af5c-a05f0602148c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.322298 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.322252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzpp2\" (UniqueName: \"kubernetes.io/projected/122e2c56-031f-4af3-af5c-a05f0602148c-kube-api-access-wzpp2\") pod \"cert-manager-cainjector-559cdd8c87-zh28z\" (UID: \"122e2c56-031f-4af3-af5c-a05f0602148c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.464595 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.464557 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" Jul 08 21:24:26.596305 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.596279 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zh28z"] Jul 08 21:24:26.598682 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:26.598661 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod122e2c56_031f_4af3_af5c_a05f0602148c.slice/crio-6e1624d1eb0b7e8ac1475d0cd1b4cbf8c9fe37b863f9d840c0e28815dae3cacd WatchSource:0}: Error finding container 6e1624d1eb0b7e8ac1475d0cd1b4cbf8c9fe37b863f9d840c0e28815dae3cacd: Status 404 returned error can't find the container with id 6e1624d1eb0b7e8ac1475d0cd1b4cbf8c9fe37b863f9d840c0e28815dae3cacd Jul 08 21:24:26.663546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.663518 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" event={"ID":"122e2c56-031f-4af3-af5c-a05f0602148c","Type":"ContainerStarted","Data":"6e1624d1eb0b7e8ac1475d0cd1b4cbf8c9fe37b863f9d840c0e28815dae3cacd"} Jul 08 21:24:26.665313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.665285 2578 generic.go:356] "Generic (PLEG): container finished" podID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerID="e3231fe54a2093e6758bb8aa15615203bafdfd6e554dfe5bfd9bb42e8021ef85" exitCode=0 Jul 08 21:24:26.665419 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:26.665344 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" event={"ID":"5970bc5a-0097-4d15-b3a3-0c4ce9a68329","Type":"ContainerDied","Data":"e3231fe54a2093e6758bb8aa15615203bafdfd6e554dfe5bfd9bb42e8021ef85"} Jul 08 21:24:27.674020 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:27.673910 2578 generic.go:356] "Generic (PLEG): container finished" podID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerID="d5d263699095b4711bd051cf08872b7022cc1e6835528c10160da5cd2ca8a2fb" exitCode=0 Jul 08 21:24:27.674020 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:27.674001 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" event={"ID":"5970bc5a-0097-4d15-b3a3-0c4ce9a68329","Type":"ContainerDied","Data":"d5d263699095b4711bd051cf08872b7022cc1e6835528c10160da5cd2ca8a2fb"} Jul 08 21:24:29.247438 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.247412 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:29.341708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.341685 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cndhk\" (UniqueName: \"kubernetes.io/projected/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-kube-api-access-cndhk\") pod \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " Jul 08 21:24:29.341838 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.341753 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-util\") pod \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " Jul 08 21:24:29.341838 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.341804 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-bundle\") pod \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\" (UID: \"5970bc5a-0097-4d15-b3a3-0c4ce9a68329\") " Jul 08 21:24:29.342267 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.342242 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-bundle" (OuterVolumeSpecName: "bundle") pod "5970bc5a-0097-4d15-b3a3-0c4ce9a68329" (UID: "5970bc5a-0097-4d15-b3a3-0c4ce9a68329"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:24:29.344149 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.344117 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-kube-api-access-cndhk" (OuterVolumeSpecName: "kube-api-access-cndhk") pod "5970bc5a-0097-4d15-b3a3-0c4ce9a68329" (UID: "5970bc5a-0097-4d15-b3a3-0c4ce9a68329"). InnerVolumeSpecName "kube-api-access-cndhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:24:29.349726 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.349699 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-util" (OuterVolumeSpecName: "util") pod "5970bc5a-0097-4d15-b3a3-0c4ce9a68329" (UID: "5970bc5a-0097-4d15-b3a3-0c4ce9a68329"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:24:29.443182 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.443149 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cndhk\" (UniqueName: \"kubernetes.io/projected/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-kube-api-access-cndhk\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:29.443182 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.443175 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:29.443182 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.443184 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5970bc5a-0097-4d15-b3a3-0c4ce9a68329-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:29.685080 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.685053 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" Jul 08 21:24:29.685245 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.685087 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fndzj4" event={"ID":"5970bc5a-0097-4d15-b3a3-0c4ce9a68329","Type":"ContainerDied","Data":"c3e7c8daf4c583ca35573bb368968dfc9d43f60d1ec2e3ef029a3188fa81bae9"} Jul 08 21:24:29.685245 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.685133 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3e7c8daf4c583ca35573bb368968dfc9d43f60d1ec2e3ef029a3188fa81bae9" Jul 08 21:24:29.686653 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.686620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" event={"ID":"122e2c56-031f-4af3-af5c-a05f0602148c","Type":"ContainerStarted","Data":"182ecb8ab6f4aa504b8858f2900824d3fe2e59f5d603c6909e758b02ab46d381"} Jul 08 21:24:29.701583 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:29.701544 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zh28z" podStartSLOduration=1.036976461 podStartE2EDuration="3.701530888s" totalImagesPullingTime="2.664554427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:24:29.700590817 +0000 UTC m=+527.611028238" watchObservedRunningTime="2026-07-08 21:24:29.701530888 +0000 UTC m=+527.611968305" Jul 08 21:24:30.547674 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.547639 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wlrdw"] Jul 08 21:24:30.548054 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548010 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="util" Jul 08 21:24:30.548054 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548024 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="util" Jul 08 21:24:30.548054 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548031 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="pull" Jul 08 21:24:30.548054 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548037 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="pull" Jul 08 21:24:30.548054 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548055 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="extract" Jul 08 21:24:30.548206 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548060 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="extract" Jul 08 21:24:30.548206 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.548117 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5970bc5a-0097-4d15-b3a3-0c4ce9a68329" containerName="extract" Jul 08 21:24:30.550522 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.550508 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.553055 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.553030 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-97xjj\"" Jul 08 21:24:30.562745 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.562694 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wlrdw"] Jul 08 21:24:30.653710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.653681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bschm\" (UniqueName: \"kubernetes.io/projected/363491e4-312d-44df-854f-90c58f702d6c-kube-api-access-bschm\") pod \"cert-manager-webhook-6486ffd997-wlrdw\" (UID: \"363491e4-312d-44df-854f-90c58f702d6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.653858 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.653797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/363491e4-312d-44df-854f-90c58f702d6c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wlrdw\" (UID: \"363491e4-312d-44df-854f-90c58f702d6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.754723 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.754697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/363491e4-312d-44df-854f-90c58f702d6c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wlrdw\" (UID: \"363491e4-312d-44df-854f-90c58f702d6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.754937 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.754742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bschm\" (UniqueName: \"kubernetes.io/projected/363491e4-312d-44df-854f-90c58f702d6c-kube-api-access-bschm\") pod \"cert-manager-webhook-6486ffd997-wlrdw\" (UID: \"363491e4-312d-44df-854f-90c58f702d6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.762589 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.762562 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/363491e4-312d-44df-854f-90c58f702d6c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wlrdw\" (UID: \"363491e4-312d-44df-854f-90c58f702d6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.762695 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.762633 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bschm\" (UniqueName: \"kubernetes.io/projected/363491e4-312d-44df-854f-90c58f702d6c-kube-api-access-bschm\") pod \"cert-manager-webhook-6486ffd997-wlrdw\" (UID: \"363491e4-312d-44df-854f-90c58f702d6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.860000 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.859929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:30.985907 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:30.985858 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wlrdw"] Jul 08 21:24:30.988424 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:30.988396 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod363491e4_312d_44df_854f_90c58f702d6c.slice/crio-2da79aec640b8620870650a35396ef1d06bcf71f73f0f6d9d181ec48fd26f406 WatchSource:0}: Error finding container 2da79aec640b8620870650a35396ef1d06bcf71f73f0f6d9d181ec48fd26f406: Status 404 returned error can't find the container with id 2da79aec640b8620870650a35396ef1d06bcf71f73f0f6d9d181ec48fd26f406 Jul 08 21:24:31.697445 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:31.697407 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" event={"ID":"363491e4-312d-44df-854f-90c58f702d6c","Type":"ContainerStarted","Data":"e6f74edbfa5d80455412dc7e876636db237b132d8b5cd16d5c3b82cf5be892e7"} Jul 08 21:24:31.697445 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:31.697448 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" event={"ID":"363491e4-312d-44df-854f-90c58f702d6c","Type":"ContainerStarted","Data":"2da79aec640b8620870650a35396ef1d06bcf71f73f0f6d9d181ec48fd26f406"} Jul 08 21:24:31.697950 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:31.697467 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:31.714795 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:31.714754 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" podStartSLOduration=1.71473885 podStartE2EDuration="1.71473885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:24:31.713143194 +0000 UTC m=+529.623580611" watchObservedRunningTime="2026-07-08 21:24:31.71473885 +0000 UTC m=+529.625176266" Jul 08 21:24:34.055312 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.055231 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bntcw"] Jul 08 21:24:34.057631 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.057613 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.060153 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.060140 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ls54h\"" Jul 08 21:24:34.067205 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.067184 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bntcw"] Jul 08 21:24:34.188044 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.188013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nq7x\" (UniqueName: \"kubernetes.io/projected/368113e5-df17-4f15-9174-48f3dae50a99-kube-api-access-2nq7x\") pod \"cert-manager-55b4d954d5-bntcw\" (UID: \"368113e5-df17-4f15-9174-48f3dae50a99\") " pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.188180 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.188061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/368113e5-df17-4f15-9174-48f3dae50a99-bound-sa-token\") pod \"cert-manager-55b4d954d5-bntcw\" (UID: \"368113e5-df17-4f15-9174-48f3dae50a99\") " pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.288639 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.288608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nq7x\" (UniqueName: \"kubernetes.io/projected/368113e5-df17-4f15-9174-48f3dae50a99-kube-api-access-2nq7x\") pod \"cert-manager-55b4d954d5-bntcw\" (UID: \"368113e5-df17-4f15-9174-48f3dae50a99\") " pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.288765 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.288658 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/368113e5-df17-4f15-9174-48f3dae50a99-bound-sa-token\") pod \"cert-manager-55b4d954d5-bntcw\" (UID: \"368113e5-df17-4f15-9174-48f3dae50a99\") " pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.296945 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.296921 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nq7x\" (UniqueName: \"kubernetes.io/projected/368113e5-df17-4f15-9174-48f3dae50a99-kube-api-access-2nq7x\") pod \"cert-manager-55b4d954d5-bntcw\" (UID: \"368113e5-df17-4f15-9174-48f3dae50a99\") " pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.297473 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.297457 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/368113e5-df17-4f15-9174-48f3dae50a99-bound-sa-token\") pod \"cert-manager-55b4d954d5-bntcw\" (UID: \"368113e5-df17-4f15-9174-48f3dae50a99\") " pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.368411 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.368339 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bntcw" Jul 08 21:24:34.496014 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.495992 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bntcw"] Jul 08 21:24:34.499148 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:34.499122 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod368113e5_df17_4f15_9174_48f3dae50a99.slice/crio-e7ca9067cced89148e889285810baadf3681b3cc5fc281730b022df03fe51419 WatchSource:0}: Error finding container e7ca9067cced89148e889285810baadf3681b3cc5fc281730b022df03fe51419: Status 404 returned error can't find the container with id e7ca9067cced89148e889285810baadf3681b3cc5fc281730b022df03fe51419 Jul 08 21:24:34.712027 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.711997 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bntcw" event={"ID":"368113e5-df17-4f15-9174-48f3dae50a99","Type":"ContainerStarted","Data":"035679d46d30caa31631c13a5c26a0076809ea27b6d056d1595a79b63623ca3d"} Jul 08 21:24:34.712187 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.712036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bntcw" event={"ID":"368113e5-df17-4f15-9174-48f3dae50a99","Type":"ContainerStarted","Data":"e7ca9067cced89148e889285810baadf3681b3cc5fc281730b022df03fe51419"} Jul 08 21:24:34.730168 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:34.730107 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-bntcw" podStartSLOduration=0.730088739 podStartE2EDuration="730.088739ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:24:34.727800643 +0000 UTC m=+532.638238076" watchObservedRunningTime="2026-07-08 21:24:34.730088739 +0000 UTC m=+532.640526158" Jul 08 21:24:37.703938 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:37.703908 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wlrdw" Jul 08 21:24:45.783075 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.783043 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2"] Jul 08 21:24:45.802968 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.802937 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2"] Jul 08 21:24:45.803136 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.803031 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.805771 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.805741 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 21:24:45.805771 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.805746 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-crzgl\"" Jul 08 21:24:45.806835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.806821 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 21:24:45.887363 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.887335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.887494 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.887365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.887494 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.887456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hsbb\" (UniqueName: \"kubernetes.io/projected/afc0e730-e840-45eb-97f6-4b8d221b72a2-kube-api-access-7hsbb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.988673 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.988647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.988853 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.988678 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.988853 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.988756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hsbb\" (UniqueName: \"kubernetes.io/projected/afc0e730-e840-45eb-97f6-4b8d221b72a2-kube-api-access-7hsbb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.989114 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.989094 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.989151 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.989101 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:45.998208 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:45.998186 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hsbb\" (UniqueName: \"kubernetes.io/projected/afc0e730-e840-45eb-97f6-4b8d221b72a2-kube-api-access-7hsbb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:46.113105 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:46.113036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:46.244569 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:46.244451 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2"] Jul 08 21:24:46.246762 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:46.246738 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc0e730_e840_45eb_97f6_4b8d221b72a2.slice/crio-43caf0e1882f8dcc84b3ff65d5ecff57ff8ffedcb90d79de7e9f8d2dff6c01d1 WatchSource:0}: Error finding container 43caf0e1882f8dcc84b3ff65d5ecff57ff8ffedcb90d79de7e9f8d2dff6c01d1: Status 404 returned error can't find the container with id 43caf0e1882f8dcc84b3ff65d5ecff57ff8ffedcb90d79de7e9f8d2dff6c01d1 Jul 08 21:24:46.768026 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:46.767995 2578 generic.go:356] "Generic (PLEG): container finished" podID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerID="923c310641652e6c16f2f427574b49ac5cb7d7dee5ac81133ab9eaba3d03d976" exitCode=0 Jul 08 21:24:46.768192 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:46.768082 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" event={"ID":"afc0e730-e840-45eb-97f6-4b8d221b72a2","Type":"ContainerDied","Data":"923c310641652e6c16f2f427574b49ac5cb7d7dee5ac81133ab9eaba3d03d976"} Jul 08 21:24:46.768192 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:46.768133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" event={"ID":"afc0e730-e840-45eb-97f6-4b8d221b72a2","Type":"ContainerStarted","Data":"43caf0e1882f8dcc84b3ff65d5ecff57ff8ffedcb90d79de7e9f8d2dff6c01d1"} Jul 08 21:24:47.775030 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:47.774994 2578 generic.go:356] "Generic (PLEG): container finished" podID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerID="5c9145e0eb3efb7c7ee55ee4359b5860677954f39ab9904f8a46dc1f88afd677" exitCode=0 Jul 08 21:24:47.775492 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:47.775056 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" event={"ID":"afc0e730-e840-45eb-97f6-4b8d221b72a2","Type":"ContainerDied","Data":"5c9145e0eb3efb7c7ee55ee4359b5860677954f39ab9904f8a46dc1f88afd677"} Jul 08 21:24:48.782023 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:48.781993 2578 generic.go:356] "Generic (PLEG): container finished" podID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerID="54ad9e061bad420e68b42849ab1b902dfcb17b91f0a47b00d4f90295a713abb5" exitCode=0 Jul 08 21:24:48.782426 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:48.782062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" event={"ID":"afc0e730-e840-45eb-97f6-4b8d221b72a2","Type":"ContainerDied","Data":"54ad9e061bad420e68b42849ab1b902dfcb17b91f0a47b00d4f90295a713abb5"} Jul 08 21:24:49.916738 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:49.916716 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:50.023067 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.023034 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-util\") pod \"afc0e730-e840-45eb-97f6-4b8d221b72a2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " Jul 08 21:24:50.023252 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.023097 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hsbb\" (UniqueName: \"kubernetes.io/projected/afc0e730-e840-45eb-97f6-4b8d221b72a2-kube-api-access-7hsbb\") pod \"afc0e730-e840-45eb-97f6-4b8d221b72a2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " Jul 08 21:24:50.023252 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.023122 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-bundle\") pod \"afc0e730-e840-45eb-97f6-4b8d221b72a2\" (UID: \"afc0e730-e840-45eb-97f6-4b8d221b72a2\") " Jul 08 21:24:50.023801 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.023780 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-bundle" (OuterVolumeSpecName: "bundle") pod "afc0e730-e840-45eb-97f6-4b8d221b72a2" (UID: "afc0e730-e840-45eb-97f6-4b8d221b72a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:24:50.025250 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.025231 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc0e730-e840-45eb-97f6-4b8d221b72a2-kube-api-access-7hsbb" (OuterVolumeSpecName: "kube-api-access-7hsbb") pod "afc0e730-e840-45eb-97f6-4b8d221b72a2" (UID: "afc0e730-e840-45eb-97f6-4b8d221b72a2"). InnerVolumeSpecName "kube-api-access-7hsbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:24:50.028573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.028550 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-util" (OuterVolumeSpecName: "util") pod "afc0e730-e840-45eb-97f6-4b8d221b72a2" (UID: "afc0e730-e840-45eb-97f6-4b8d221b72a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:24:50.124313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.124240 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hsbb\" (UniqueName: \"kubernetes.io/projected/afc0e730-e840-45eb-97f6-4b8d221b72a2-kube-api-access-7hsbb\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:50.124313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.124266 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:50.124313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.124276 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/afc0e730-e840-45eb-97f6-4b8d221b72a2-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:24:50.792473 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.792446 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" Jul 08 21:24:50.792606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.792444 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqkps2" event={"ID":"afc0e730-e840-45eb-97f6-4b8d221b72a2","Type":"ContainerDied","Data":"43caf0e1882f8dcc84b3ff65d5ecff57ff8ffedcb90d79de7e9f8d2dff6c01d1"} Jul 08 21:24:50.792606 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:50.792590 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43caf0e1882f8dcc84b3ff65d5ecff57ff8ffedcb90d79de7e9f8d2dff6c01d1" Jul 08 21:24:57.328045 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328007 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh"] Jul 08 21:24:57.328650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328518 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="util" Jul 08 21:24:57.328650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328536 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="util" Jul 08 21:24:57.328650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328574 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="extract" Jul 08 21:24:57.328650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328583 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="extract" Jul 08 21:24:57.328650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328597 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="pull" Jul 08 21:24:57.328650 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328605 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="pull" Jul 08 21:24:57.329011 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.328793 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="afc0e730-e840-45eb-97f6-4b8d221b72a2" containerName="extract" Jul 08 21:24:57.333827 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.333805 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.336738 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.336718 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 21:24:57.336862 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.336749 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 21:24:57.336968 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.336910 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 21:24:57.337030 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.336999 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k5x2r\"" Jul 08 21:24:57.337152 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.337136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 21:24:57.342591 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.342569 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh"] Jul 08 21:24:57.487545 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.487512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-manifests\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.487700 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.487552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-tmp\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.487700 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.487573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-webhook-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.487700 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.487653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44jdh\" (UniqueName: \"kubernetes.io/projected/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-kube-api-access-44jdh\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.487804 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.487727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.589103 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.589027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-webhook-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.589103 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.589066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44jdh\" (UniqueName: \"kubernetes.io/projected/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-kube-api-access-44jdh\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.589294 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.589104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.589294 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.589127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-manifests\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.589294 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.589155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-tmp\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.589551 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.589532 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-manifests\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.591421 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.591393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-tmp\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.591757 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.591740 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.591805 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.591787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-webhook-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.598005 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.597979 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44jdh\" (UniqueName: \"kubernetes.io/projected/b15932d2-1e45-43d2-a3eb-c0bcfb85746d-kube-api-access-44jdh\") pod \"opendatahub-operator-controller-manager-c9df76854-wpvbh\" (UID: \"b15932d2-1e45-43d2-a3eb-c0bcfb85746d\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.644757 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.644731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:24:57.777972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.777943 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh"] Jul 08 21:24:57.781127 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:24:57.781098 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15932d2_1e45_43d2_a3eb_c0bcfb85746d.slice/crio-a8d790152120c918117551faef338f768af16115a2887f849f84c5e743a836fd WatchSource:0}: Error finding container a8d790152120c918117551faef338f768af16115a2887f849f84c5e743a836fd: Status 404 returned error can't find the container with id a8d790152120c918117551faef338f768af16115a2887f849f84c5e743a836fd Jul 08 21:24:57.829676 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:24:57.829646 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" event={"ID":"b15932d2-1e45-43d2-a3eb-c0bcfb85746d","Type":"ContainerStarted","Data":"a8d790152120c918117551faef338f768af16115a2887f849f84c5e743a836fd"} Jul 08 21:25:01.861708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:01.861665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" event={"ID":"b15932d2-1e45-43d2-a3eb-c0bcfb85746d","Type":"ContainerStarted","Data":"abaebfbabab68949018ce20d59905c62702bddb70e622efe75486e14f7aa2f30"} Jul 08 21:25:02.868055 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:02.868022 2578 generic.go:356] "Generic (PLEG): container finished" podID="b15932d2-1e45-43d2-a3eb-c0bcfb85746d" containerID="abaebfbabab68949018ce20d59905c62702bddb70e622efe75486e14f7aa2f30" exitCode=0 Jul 08 21:25:02.868430 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:02.868109 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" event={"ID":"b15932d2-1e45-43d2-a3eb-c0bcfb85746d","Type":"ContainerDied","Data":"abaebfbabab68949018ce20d59905c62702bddb70e622efe75486e14f7aa2f30"} Jul 08 21:25:03.875287 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:03.875189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" event={"ID":"b15932d2-1e45-43d2-a3eb-c0bcfb85746d","Type":"ContainerStarted","Data":"b399bd79b644b1a0ebc6139ffd74a7d6e86f139a4d70c886fca8b78da7984f26"} Jul 08 21:25:03.875646 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:03.875311 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:25:03.898743 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:03.898688 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" podStartSLOduration=2.115387958 podStartE2EDuration="6.898672019s" totalImagesPullingTime="3.687978042s" totalInitContainerRuntime="1.095306019s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:03.896859205 +0000 UTC m=+561.807296702" watchObservedRunningTime="2026-07-08 21:25:03.898672019 +0000 UTC m=+561.809109437" Jul 08 21:25:14.882594 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:14.882562 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-wpvbh" Jul 08 21:25:16.208260 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.208222 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq"] Jul 08 21:25:16.215349 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.215327 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.219138 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.219112 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c5h42\"" Jul 08 21:25:16.219274 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.219115 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 21:25:16.219274 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.219153 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 21:25:16.219274 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.219172 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:25:16.219274 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.219118 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 21:25:16.219274 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.219222 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 21:25:16.222747 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.222727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq"] Jul 08 21:25:16.231967 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.231947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a18c2353-0bc5-49d6-a36d-cdbca143400a-manager-config\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.232063 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.231978 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a18c2353-0bc5-49d6-a36d-cdbca143400a-cert\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.232063 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.232001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a18c2353-0bc5-49d6-a36d-cdbca143400a-metrics-cert\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.232182 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.232150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4s8\" (UniqueName: \"kubernetes.io/projected/a18c2353-0bc5-49d6-a36d-cdbca143400a-kube-api-access-7p4s8\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.333409 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.333380 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4s8\" (UniqueName: \"kubernetes.io/projected/a18c2353-0bc5-49d6-a36d-cdbca143400a-kube-api-access-7p4s8\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.333556 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.333418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a18c2353-0bc5-49d6-a36d-cdbca143400a-manager-config\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.333556 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.333441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a18c2353-0bc5-49d6-a36d-cdbca143400a-cert\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.333556 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.333474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a18c2353-0bc5-49d6-a36d-cdbca143400a-metrics-cert\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.334190 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.334127 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a18c2353-0bc5-49d6-a36d-cdbca143400a-manager-config\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.336142 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.336118 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a18c2353-0bc5-49d6-a36d-cdbca143400a-cert\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.336230 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.336162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a18c2353-0bc5-49d6-a36d-cdbca143400a-metrics-cert\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.361325 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.361302 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4s8\" (UniqueName: \"kubernetes.io/projected/a18c2353-0bc5-49d6-a36d-cdbca143400a-kube-api-access-7p4s8\") pod \"lws-controller-manager-5cdf98c45c-5b5hq\" (UID: \"a18c2353-0bc5-49d6-a36d-cdbca143400a\") " pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.525358 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.525288 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:16.681428 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.681397 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq"] Jul 08 21:25:16.682862 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:25:16.682837 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18c2353_0bc5_49d6_a36d_cdbca143400a.slice/crio-a9550d44c74fe015ec055f7265a7ab294e0f7db5bff88f893191b1e5bd80d264 WatchSource:0}: Error finding container a9550d44c74fe015ec055f7265a7ab294e0f7db5bff88f893191b1e5bd80d264: Status 404 returned error can't find the container with id a9550d44c74fe015ec055f7265a7ab294e0f7db5bff88f893191b1e5bd80d264 Jul 08 21:25:16.939321 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:16.939288 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" event={"ID":"a18c2353-0bc5-49d6-a36d-cdbca143400a","Type":"ContainerStarted","Data":"a9550d44c74fe015ec055f7265a7ab294e0f7db5bff88f893191b1e5bd80d264"} Jul 08 21:25:17.117430 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.117398 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92"] Jul 08 21:25:17.121956 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.121937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.125702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.125676 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 21:25:17.131388 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.126411 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 21:25:17.131936 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.131919 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 21:25:17.132007 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.131957 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 21:25:17.132007 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.131980 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gkl8x\"" Jul 08 21:25:17.140097 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.140077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-tls-certs\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.140193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.140130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-tmp\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.140193 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.140156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9rh\" (UniqueName: \"kubernetes.io/projected/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-kube-api-access-nw9rh\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.144422 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.144399 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92"] Jul 08 21:25:17.240957 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.240859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-tls-certs\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.240957 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.240938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-tmp\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.241446 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.240959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9rh\" (UniqueName: \"kubernetes.io/projected/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-kube-api-access-nw9rh\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.243698 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.243670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-tmp\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.243840 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.243822 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-tls-certs\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.255764 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.255741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9rh\" (UniqueName: \"kubernetes.io/projected/aa85c81a-9b7c-4661-b0cb-9ab227f37d7f-kube-api-access-nw9rh\") pod \"kube-auth-proxy-64b5cdf9f6-s8p92\" (UID: \"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.438422 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.438180 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" Jul 08 21:25:17.604771 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.604740 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92"] Jul 08 21:25:17.606764 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:25:17.606733 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa85c81a_9b7c_4661_b0cb_9ab227f37d7f.slice/crio-ac2f849f155c306734d79fb06a936ace11804b688cf6a5c906f01a0e57a36d55 WatchSource:0}: Error finding container ac2f849f155c306734d79fb06a936ace11804b688cf6a5c906f01a0e57a36d55: Status 404 returned error can't find the container with id ac2f849f155c306734d79fb06a936ace11804b688cf6a5c906f01a0e57a36d55 Jul 08 21:25:17.946313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:17.946263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" event={"ID":"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f","Type":"ContainerStarted","Data":"ac2f849f155c306734d79fb06a936ace11804b688cf6a5c906f01a0e57a36d55"} Jul 08 21:25:18.953085 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:18.953048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" event={"ID":"a18c2353-0bc5-49d6-a36d-cdbca143400a","Type":"ContainerStarted","Data":"40f4f0c1ccc4057fd134a4f707d5c7f375518790b759535201c589fc3fe0cab0"} Jul 08 21:25:18.953547 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:18.953184 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:18.972447 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:18.972399 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" podStartSLOduration=1.5675126160000001 podStartE2EDuration="2.972383179s" totalImagesPullingTime="1.404870563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:18.971373511 +0000 UTC m=+576.881810930" watchObservedRunningTime="2026-07-08 21:25:18.972383179 +0000 UTC m=+576.882820596" Jul 08 21:25:20.964441 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:20.964410 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" event={"ID":"aa85c81a-9b7c-4661-b0cb-9ab227f37d7f","Type":"ContainerStarted","Data":"f6ee68e5f3c7d1d32a64ee8c6907027000906c27b83505364d0624529699a875"} Jul 08 21:25:20.983487 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:20.983386 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-s8p92" podStartSLOduration=0.863560791 podStartE2EDuration="3.983366622s" totalImagesPullingTime="3.119805831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:20.980772431 +0000 UTC m=+578.891209849" watchObservedRunningTime="2026-07-08 21:25:20.983366622 +0000 UTC m=+578.893804040" Jul 08 21:25:29.960360 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:29.960330 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5cdf98c45c-5b5hq" Jul 08 21:25:42.531487 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:42.531455 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:25:42.533813 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:25:42.533794 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:26:33.508190 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.508156 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7"] Jul 08 21:26:33.511233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.511217 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.513750 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.513729 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 21:26:33.514773 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.514756 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dqkvb\"" Jul 08 21:26:33.514888 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.514776 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 21:26:33.519383 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.519363 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7"] Jul 08 21:26:33.601110 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.601083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.601234 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.601123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7drd\" (UniqueName: \"kubernetes.io/projected/96867611-09a8-44bf-91ea-87c2603d6cdc-kube-api-access-t7drd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.601234 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.601183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.702210 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.702173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.702328 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.702223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7drd\" (UniqueName: \"kubernetes.io/projected/96867611-09a8-44bf-91ea-87c2603d6cdc-kube-api-access-t7drd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.702328 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.702257 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.702527 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.702508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.702567 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.702545 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.711094 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.711071 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7drd\" (UniqueName: \"kubernetes.io/projected/96867611-09a8-44bf-91ea-87c2603d6cdc-kube-api-access-t7drd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.821830 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.821756 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:33.962784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:33.962763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7"] Jul 08 21:26:33.964691 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:26:33.964664 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96867611_09a8_44bf_91ea_87c2603d6cdc.slice/crio-0a84257b2459c845d0ab252b48310662645f13ed994ffb04e59bf116db3a48a6 WatchSource:0}: Error finding container 0a84257b2459c845d0ab252b48310662645f13ed994ffb04e59bf116db3a48a6: Status 404 returned error can't find the container with id 0a84257b2459c845d0ab252b48310662645f13ed994ffb04e59bf116db3a48a6 Jul 08 21:26:34.108372 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.108295 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs"] Jul 08 21:26:34.111100 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.111074 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.123837 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.123174 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs"] Jul 08 21:26:34.212612 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.212581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.212612 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.212615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mphr6\" (UniqueName: \"kubernetes.io/projected/eaabac55-ab9d-46d3-885c-5f50e283c6df-kube-api-access-mphr6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.212794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.212637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.313296 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.313266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.313296 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.313299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mphr6\" (UniqueName: \"kubernetes.io/projected/eaabac55-ab9d-46d3-885c-5f50e283c6df-kube-api-access-mphr6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.313490 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.313322 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.313730 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.313709 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.313790 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.313727 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.321110 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.321089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mphr6\" (UniqueName: \"kubernetes.io/projected/eaabac55-ab9d-46d3-885c-5f50e283c6df-kube-api-access-mphr6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.321664 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.321640 2578 generic.go:356] "Generic (PLEG): container finished" podID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerID="f3fd2891da98dbe6628e6a427f71e5246dc01573658479408a4bfcf5168a5ed0" exitCode=0 Jul 08 21:26:34.321758 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.321721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" event={"ID":"96867611-09a8-44bf-91ea-87c2603d6cdc","Type":"ContainerDied","Data":"f3fd2891da98dbe6628e6a427f71e5246dc01573658479408a4bfcf5168a5ed0"} Jul 08 21:26:34.321808 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.321767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" event={"ID":"96867611-09a8-44bf-91ea-87c2603d6cdc","Type":"ContainerStarted","Data":"0a84257b2459c845d0ab252b48310662645f13ed994ffb04e59bf116db3a48a6"} Jul 08 21:26:34.322185 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.322168 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:26:34.431802 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.431778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:34.516076 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.509420 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c"] Jul 08 21:26:34.527096 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.526989 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c"] Jul 08 21:26:34.527176 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.527113 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.583769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.583748 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs"] Jul 08 21:26:34.585940 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:26:34.585915 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaabac55_ab9d_46d3_885c_5f50e283c6df.slice/crio-1453005c329e7b3a471c302100cb2786279fd462bddb9ca0f6126ff2a217b9a9 WatchSource:0}: Error finding container 1453005c329e7b3a471c302100cb2786279fd462bddb9ca0f6126ff2a217b9a9: Status 404 returned error can't find the container with id 1453005c329e7b3a471c302100cb2786279fd462bddb9ca0f6126ff2a217b9a9 Jul 08 21:26:34.616263 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.616087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9rm4\" (UniqueName: \"kubernetes.io/projected/aba21b3d-8986-4971-9ae8-9d877496e7ff-kube-api-access-n9rm4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.616263 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.616128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.616263 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.616188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.717121 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.717056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.717632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.717335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9rm4\" (UniqueName: \"kubernetes.io/projected/aba21b3d-8986-4971-9ae8-9d877496e7ff-kube-api-access-n9rm4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.717632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.717402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.717632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.717491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.717784 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.717716 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.725051 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.725032 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9rm4\" (UniqueName: \"kubernetes.io/projected/aba21b3d-8986-4971-9ae8-9d877496e7ff-kube-api-access-n9rm4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.838762 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.838735 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:34.912819 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.912794 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69"] Jul 08 21:26:34.915504 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.915474 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:34.925657 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.925344 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69"] Jul 08 21:26:34.983561 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:34.983538 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c"] Jul 08 21:26:34.985722 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:26:34.985695 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba21b3d_8986_4971_9ae8_9d877496e7ff.slice/crio-e49f02b00e249801712fd77c423b5bb1135eda4c48e495007213d34b32044f9f WatchSource:0}: Error finding container e49f02b00e249801712fd77c423b5bb1135eda4c48e495007213d34b32044f9f: Status 404 returned error can't find the container with id e49f02b00e249801712fd77c423b5bb1135eda4c48e495007213d34b32044f9f Jul 08 21:26:35.020227 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.020201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.020312 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.020238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbcq\" (UniqueName: \"kubernetes.io/projected/cbd54043-12f9-4647-8b97-48586f7badbd-kube-api-access-zmbcq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.020312 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.020259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.121173 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.121144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.121352 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.121182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbcq\" (UniqueName: \"kubernetes.io/projected/cbd54043-12f9-4647-8b97-48586f7badbd-kube-api-access-zmbcq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.121352 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.121207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.121584 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.121557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.121649 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.121566 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.129730 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.129700 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbcq\" (UniqueName: \"kubernetes.io/projected/cbd54043-12f9-4647-8b97-48586f7badbd-kube-api-access-zmbcq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.226348 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.226323 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:35.332313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.332284 2578 generic.go:356] "Generic (PLEG): container finished" podID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerID="fb126854ef357418f30131b619f63042a3cf8117e0a006b9251e874aa5a964f5" exitCode=0 Jul 08 21:26:35.332457 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.332377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" event={"ID":"eaabac55-ab9d-46d3-885c-5f50e283c6df","Type":"ContainerDied","Data":"fb126854ef357418f30131b619f63042a3cf8117e0a006b9251e874aa5a964f5"} Jul 08 21:26:35.332457 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.332442 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" event={"ID":"eaabac55-ab9d-46d3-885c-5f50e283c6df","Type":"ContainerStarted","Data":"1453005c329e7b3a471c302100cb2786279fd462bddb9ca0f6126ff2a217b9a9"} Jul 08 21:26:35.334714 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.334688 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" event={"ID":"96867611-09a8-44bf-91ea-87c2603d6cdc","Type":"ContainerStarted","Data":"8fd685aca65ce0ffc13a62ecd8a3bbc544f03f2928702613a62e40336f413bdd"} Jul 08 21:26:35.337220 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.337186 2578 generic.go:356] "Generic (PLEG): container finished" podID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerID="be1092f10c167e6f11534978024e17f530d0ec2430689521a4213c0f0e191544" exitCode=0 Jul 08 21:26:35.337326 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.337250 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" event={"ID":"aba21b3d-8986-4971-9ae8-9d877496e7ff","Type":"ContainerDied","Data":"be1092f10c167e6f11534978024e17f530d0ec2430689521a4213c0f0e191544"} Jul 08 21:26:35.337326 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.337310 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" event={"ID":"aba21b3d-8986-4971-9ae8-9d877496e7ff","Type":"ContainerStarted","Data":"e49f02b00e249801712fd77c423b5bb1135eda4c48e495007213d34b32044f9f"} Jul 08 21:26:35.364732 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:35.364710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69"] Jul 08 21:26:35.396700 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:26:35.396673 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd54043_12f9_4647_8b97_48586f7badbd.slice/crio-b8348cbb1dc19d5a1ec9c90924cf45a4cb7d59450950b15a72aecd00464f0396 WatchSource:0}: Error finding container b8348cbb1dc19d5a1ec9c90924cf45a4cb7d59450950b15a72aecd00464f0396: Status 404 returned error can't find the container with id b8348cbb1dc19d5a1ec9c90924cf45a4cb7d59450950b15a72aecd00464f0396 Jul 08 21:26:36.344425 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.344383 2578 generic.go:356] "Generic (PLEG): container finished" podID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerID="8fd685aca65ce0ffc13a62ecd8a3bbc544f03f2928702613a62e40336f413bdd" exitCode=0 Jul 08 21:26:36.344814 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.344467 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" event={"ID":"96867611-09a8-44bf-91ea-87c2603d6cdc","Type":"ContainerDied","Data":"8fd685aca65ce0ffc13a62ecd8a3bbc544f03f2928702613a62e40336f413bdd"} Jul 08 21:26:36.346383 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.346361 2578 generic.go:356] "Generic (PLEG): container finished" podID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerID="d7e11a6b6da3445b903e0e3b9020a31bc3c44c9982f1e5d45db62252a4e088d3" exitCode=0 Jul 08 21:26:36.346478 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.346435 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" event={"ID":"aba21b3d-8986-4971-9ae8-9d877496e7ff","Type":"ContainerDied","Data":"d7e11a6b6da3445b903e0e3b9020a31bc3c44c9982f1e5d45db62252a4e088d3"} Jul 08 21:26:36.355571 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.355545 2578 generic.go:356] "Generic (PLEG): container finished" podID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerID="b5db28b2249977912d5638627344f7425483c9c363fc47661f0c6e059ab095a1" exitCode=0 Jul 08 21:26:36.355677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.355636 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" event={"ID":"eaabac55-ab9d-46d3-885c-5f50e283c6df","Type":"ContainerDied","Data":"b5db28b2249977912d5638627344f7425483c9c363fc47661f0c6e059ab095a1"} Jul 08 21:26:36.357243 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.357222 2578 generic.go:356] "Generic (PLEG): container finished" podID="cbd54043-12f9-4647-8b97-48586f7badbd" containerID="c6ab412f11283fdcdbb2874a8ab292473117773aa9c34bbb9297e41ae3ffeb0a" exitCode=0 Jul 08 21:26:36.357299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.357263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" event={"ID":"cbd54043-12f9-4647-8b97-48586f7badbd","Type":"ContainerDied","Data":"c6ab412f11283fdcdbb2874a8ab292473117773aa9c34bbb9297e41ae3ffeb0a"} Jul 08 21:26:36.357347 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:36.357301 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" event={"ID":"cbd54043-12f9-4647-8b97-48586f7badbd","Type":"ContainerStarted","Data":"b8348cbb1dc19d5a1ec9c90924cf45a4cb7d59450950b15a72aecd00464f0396"} Jul 08 21:26:37.364336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.364305 2578 generic.go:356] "Generic (PLEG): container finished" podID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerID="ed62da77335f31b2852616d11044a29567c9f6dcb71e154b5b0c0ab27fe4cd73" exitCode=0 Jul 08 21:26:37.364687 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.364385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" event={"ID":"96867611-09a8-44bf-91ea-87c2603d6cdc","Type":"ContainerDied","Data":"ed62da77335f31b2852616d11044a29567c9f6dcb71e154b5b0c0ab27fe4cd73"} Jul 08 21:26:37.366492 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.366468 2578 generic.go:356] "Generic (PLEG): container finished" podID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerID="3bf61bd94c358509b19485cefddab2b7d4a73a5fe82a526e09137fef66e1ea8d" exitCode=0 Jul 08 21:26:37.366576 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.366553 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" event={"ID":"aba21b3d-8986-4971-9ae8-9d877496e7ff","Type":"ContainerDied","Data":"3bf61bd94c358509b19485cefddab2b7d4a73a5fe82a526e09137fef66e1ea8d"} Jul 08 21:26:37.368508 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.368491 2578 generic.go:356] "Generic (PLEG): container finished" podID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerID="903dd3b71bfdd70ef062db971456520bebc1df047e507f44623af2e9756746cd" exitCode=0 Jul 08 21:26:37.368581 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.368558 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" event={"ID":"eaabac55-ab9d-46d3-885c-5f50e283c6df","Type":"ContainerDied","Data":"903dd3b71bfdd70ef062db971456520bebc1df047e507f44623af2e9756746cd"} Jul 08 21:26:37.370219 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.370200 2578 generic.go:356] "Generic (PLEG): container finished" podID="cbd54043-12f9-4647-8b97-48586f7badbd" containerID="2f13151c8145187b222b732bce6aebd50b633c10fae2a8dffff4ceb5e5117a4d" exitCode=0 Jul 08 21:26:37.370296 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:37.370225 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" event={"ID":"cbd54043-12f9-4647-8b97-48586f7badbd","Type":"ContainerDied","Data":"2f13151c8145187b222b732bce6aebd50b633c10fae2a8dffff4ceb5e5117a4d"} Jul 08 21:26:38.378448 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.378413 2578 generic.go:356] "Generic (PLEG): container finished" podID="cbd54043-12f9-4647-8b97-48586f7badbd" containerID="4b2fb39e2515459efae4453760a5a304e02ac73d0cec86f1d570e1154e5c4091" exitCode=0 Jul 08 21:26:38.378812 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.378489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" event={"ID":"cbd54043-12f9-4647-8b97-48586f7badbd","Type":"ContainerDied","Data":"4b2fb39e2515459efae4453760a5a304e02ac73d0cec86f1d570e1154e5c4091"} Jul 08 21:26:38.527502 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.527478 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:38.562774 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.562754 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:38.566335 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.566312 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:38.653047 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.652972 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-util\") pod \"eaabac55-ab9d-46d3-885c-5f50e283c6df\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " Jul 08 21:26:38.653047 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653011 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-bundle\") pod \"96867611-09a8-44bf-91ea-87c2603d6cdc\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " Jul 08 21:26:38.653047 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653033 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9rm4\" (UniqueName: \"kubernetes.io/projected/aba21b3d-8986-4971-9ae8-9d877496e7ff-kube-api-access-n9rm4\") pod \"aba21b3d-8986-4971-9ae8-9d877496e7ff\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " Jul 08 21:26:38.653302 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653066 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-bundle\") pod \"aba21b3d-8986-4971-9ae8-9d877496e7ff\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " Jul 08 21:26:38.653302 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653092 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mphr6\" (UniqueName: \"kubernetes.io/projected/eaabac55-ab9d-46d3-885c-5f50e283c6df-kube-api-access-mphr6\") pod \"eaabac55-ab9d-46d3-885c-5f50e283c6df\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " Jul 08 21:26:38.653302 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653130 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-util\") pod \"aba21b3d-8986-4971-9ae8-9d877496e7ff\" (UID: \"aba21b3d-8986-4971-9ae8-9d877496e7ff\") " Jul 08 21:26:38.653302 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653183 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-util\") pod \"96867611-09a8-44bf-91ea-87c2603d6cdc\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " Jul 08 21:26:38.653302 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653212 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7drd\" (UniqueName: \"kubernetes.io/projected/96867611-09a8-44bf-91ea-87c2603d6cdc-kube-api-access-t7drd\") pod \"96867611-09a8-44bf-91ea-87c2603d6cdc\" (UID: \"96867611-09a8-44bf-91ea-87c2603d6cdc\") " Jul 08 21:26:38.653649 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653606 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-bundle" (OuterVolumeSpecName: "bundle") pod "aba21b3d-8986-4971-9ae8-9d877496e7ff" (UID: "aba21b3d-8986-4971-9ae8-9d877496e7ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:38.653788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653648 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-bundle" (OuterVolumeSpecName: "bundle") pod "96867611-09a8-44bf-91ea-87c2603d6cdc" (UID: "96867611-09a8-44bf-91ea-87c2603d6cdc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:38.653788 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.653776 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-bundle\") pod \"eaabac55-ab9d-46d3-885c-5f50e283c6df\" (UID: \"eaabac55-ab9d-46d3-885c-5f50e283c6df\") " Jul 08 21:26:38.654577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.654091 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.654577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.654110 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.654577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.654524 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-bundle" (OuterVolumeSpecName: "bundle") pod "eaabac55-ab9d-46d3-885c-5f50e283c6df" (UID: "eaabac55-ab9d-46d3-885c-5f50e283c6df"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:38.656045 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.655673 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aba21b3d-8986-4971-9ae8-9d877496e7ff-kube-api-access-n9rm4" (OuterVolumeSpecName: "kube-api-access-n9rm4") pod "aba21b3d-8986-4971-9ae8-9d877496e7ff" (UID: "aba21b3d-8986-4971-9ae8-9d877496e7ff"). InnerVolumeSpecName "kube-api-access-n9rm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:26:38.656045 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.655759 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaabac55-ab9d-46d3-885c-5f50e283c6df-kube-api-access-mphr6" (OuterVolumeSpecName: "kube-api-access-mphr6") pod "eaabac55-ab9d-46d3-885c-5f50e283c6df" (UID: "eaabac55-ab9d-46d3-885c-5f50e283c6df"). InnerVolumeSpecName "kube-api-access-mphr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:26:38.656206 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.656187 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96867611-09a8-44bf-91ea-87c2603d6cdc-kube-api-access-t7drd" (OuterVolumeSpecName: "kube-api-access-t7drd") pod "96867611-09a8-44bf-91ea-87c2603d6cdc" (UID: "96867611-09a8-44bf-91ea-87c2603d6cdc"). InnerVolumeSpecName "kube-api-access-t7drd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:26:38.659378 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.659352 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-util" (OuterVolumeSpecName: "util") pod "aba21b3d-8986-4971-9ae8-9d877496e7ff" (UID: "aba21b3d-8986-4971-9ae8-9d877496e7ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:38.659688 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.659672 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-util" (OuterVolumeSpecName: "util") pod "eaabac55-ab9d-46d3-885c-5f50e283c6df" (UID: "eaabac55-ab9d-46d3-885c-5f50e283c6df"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:38.660158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.660140 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-util" (OuterVolumeSpecName: "util") pod "96867611-09a8-44bf-91ea-87c2603d6cdc" (UID: "96867611-09a8-44bf-91ea-87c2603d6cdc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:38.755423 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755399 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.755423 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755420 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9rm4\" (UniqueName: \"kubernetes.io/projected/aba21b3d-8986-4971-9ae8-9d877496e7ff-kube-api-access-n9rm4\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.755562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755432 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mphr6\" (UniqueName: \"kubernetes.io/projected/eaabac55-ab9d-46d3-885c-5f50e283c6df-kube-api-access-mphr6\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.755562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755441 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aba21b3d-8986-4971-9ae8-9d877496e7ff-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.755562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755449 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96867611-09a8-44bf-91ea-87c2603d6cdc-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.755562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755456 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7drd\" (UniqueName: \"kubernetes.io/projected/96867611-09a8-44bf-91ea-87c2603d6cdc-kube-api-access-t7drd\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:38.755562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:38.755464 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaabac55-ab9d-46d3-885c-5f50e283c6df-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:39.385844 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.385812 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" Jul 08 21:26:39.385844 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.385824 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs" event={"ID":"eaabac55-ab9d-46d3-885c-5f50e283c6df","Type":"ContainerDied","Data":"1453005c329e7b3a471c302100cb2786279fd462bddb9ca0f6126ff2a217b9a9"} Jul 08 21:26:39.386283 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.385895 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1453005c329e7b3a471c302100cb2786279fd462bddb9ca0f6126ff2a217b9a9" Jul 08 21:26:39.387772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.387752 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" event={"ID":"96867611-09a8-44bf-91ea-87c2603d6cdc","Type":"ContainerDied","Data":"0a84257b2459c845d0ab252b48310662645f13ed994ffb04e59bf116db3a48a6"} Jul 08 21:26:39.387772 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.387764 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7" Jul 08 21:26:39.387951 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.387785 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a84257b2459c845d0ab252b48310662645f13ed994ffb04e59bf116db3a48a6" Jul 08 21:26:39.398010 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.397813 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" event={"ID":"aba21b3d-8986-4971-9ae8-9d877496e7ff","Type":"ContainerDied","Data":"e49f02b00e249801712fd77c423b5bb1135eda4c48e495007213d34b32044f9f"} Jul 08 21:26:39.398010 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.397913 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e49f02b00e249801712fd77c423b5bb1135eda4c48e495007213d34b32044f9f" Jul 08 21:26:39.398158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.398021 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c" Jul 08 21:26:39.523073 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.523051 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:39.661494 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.661406 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-bundle\") pod \"cbd54043-12f9-4647-8b97-48586f7badbd\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " Jul 08 21:26:39.661494 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.661450 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-util\") pod \"cbd54043-12f9-4647-8b97-48586f7badbd\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " Jul 08 21:26:39.661674 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.661526 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmbcq\" (UniqueName: \"kubernetes.io/projected/cbd54043-12f9-4647-8b97-48586f7badbd-kube-api-access-zmbcq\") pod \"cbd54043-12f9-4647-8b97-48586f7badbd\" (UID: \"cbd54043-12f9-4647-8b97-48586f7badbd\") " Jul 08 21:26:39.662100 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.662065 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-bundle" (OuterVolumeSpecName: "bundle") pod "cbd54043-12f9-4647-8b97-48586f7badbd" (UID: "cbd54043-12f9-4647-8b97-48586f7badbd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:39.663727 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.663705 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd54043-12f9-4647-8b97-48586f7badbd-kube-api-access-zmbcq" (OuterVolumeSpecName: "kube-api-access-zmbcq") pod "cbd54043-12f9-4647-8b97-48586f7badbd" (UID: "cbd54043-12f9-4647-8b97-48586f7badbd"). InnerVolumeSpecName "kube-api-access-zmbcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:26:39.669154 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.669127 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-util" (OuterVolumeSpecName: "util") pod "cbd54043-12f9-4647-8b97-48586f7badbd" (UID: "cbd54043-12f9-4647-8b97-48586f7badbd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:26:39.763008 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.762982 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-bundle\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:39.763008 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.763004 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbd54043-12f9-4647-8b97-48586f7badbd-util\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:39.763187 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:39.763014 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmbcq\" (UniqueName: \"kubernetes.io/projected/cbd54043-12f9-4647-8b97-48586f7badbd-kube-api-access-zmbcq\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:26:40.404952 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:40.404917 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" event={"ID":"cbd54043-12f9-4647-8b97-48586f7badbd","Type":"ContainerDied","Data":"b8348cbb1dc19d5a1ec9c90924cf45a4cb7d59450950b15a72aecd00464f0396"} Jul 08 21:26:40.404952 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:40.404948 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69" Jul 08 21:26:40.405376 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:26:40.404970 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8348cbb1dc19d5a1ec9c90924cf45a4cb7d59450950b15a72aecd00464f0396" Jul 08 21:27:11.039641 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.039608 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd"] Jul 08 21:27:11.040239 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040219 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="util" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040244 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="util" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040258 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="extract" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040267 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="extract" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040276 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="pull" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040281 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="pull" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040289 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="pull" Jul 08 21:27:11.040299 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040296 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="pull" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040306 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="pull" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040311 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="pull" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040323 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="util" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040328 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="util" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040335 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="util" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040340 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="util" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040347 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="util" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040351 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="util" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040358 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040363 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040368 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="pull" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040373 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="pull" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040378 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040384 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040392 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040396 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040462 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaabac55-ab9d-46d3-885c-5f50e283c6df" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040470 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96867611-09a8-44bf-91ea-87c2603d6cdc" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040475 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aba21b3d-8986-4971-9ae8-9d877496e7ff" containerName="extract" Jul 08 21:27:11.040506 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.040484 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd54043-12f9-4647-8b97-48586f7badbd" containerName="extract" Jul 08 21:27:11.043235 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.043211 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.045518 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.045496 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 21:27:11.045654 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.045638 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dqkvb\"" Jul 08 21:27:11.046553 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.046532 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 21:27:11.046625 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.046563 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 21:27:11.046679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.046654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 21:27:11.050949 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.050929 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd"] Jul 08 21:27:11.111932 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.111898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc342ffe-374f-4ae8-b458-3f911cb55fa4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.112095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.111936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc342ffe-374f-4ae8-b458-3f911cb55fa4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.112095 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.111964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z87j2\" (UniqueName: \"kubernetes.io/projected/dc342ffe-374f-4ae8-b458-3f911cb55fa4-kube-api-access-z87j2\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.212794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.212764 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc342ffe-374f-4ae8-b458-3f911cb55fa4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.212794 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.212795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc342ffe-374f-4ae8-b458-3f911cb55fa4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.213046 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.212816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z87j2\" (UniqueName: \"kubernetes.io/projected/dc342ffe-374f-4ae8-b458-3f911cb55fa4-kube-api-access-z87j2\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.213046 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:27:11.212949 2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 08 21:27:11.213046 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:27:11.213024 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc342ffe-374f-4ae8-b458-3f911cb55fa4-plugin-serving-cert podName:dc342ffe-374f-4ae8-b458-3f911cb55fa4 nodeName:}" failed. No retries permitted until 2026-07-08 21:27:11.71300241 +0000 UTC m=+689.623439815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/dc342ffe-374f-4ae8-b458-3f911cb55fa4-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-2z6gd" (UID: "dc342ffe-374f-4ae8-b458-3f911cb55fa4") : secret "plugin-serving-cert" not found Jul 08 21:27:11.213407 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.213389 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc342ffe-374f-4ae8-b458-3f911cb55fa4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.223325 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.223306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z87j2\" (UniqueName: \"kubernetes.io/projected/dc342ffe-374f-4ae8-b458-3f911cb55fa4-kube-api-access-z87j2\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.716755 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.716725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc342ffe-374f-4ae8-b458-3f911cb55fa4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.719182 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.719164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc342ffe-374f-4ae8-b458-3f911cb55fa4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2z6gd\" (UID: \"dc342ffe-374f-4ae8-b458-3f911cb55fa4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:11.953648 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:11.953606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" Jul 08 21:27:12.082699 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:12.082673 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd"] Jul 08 21:27:12.084650 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:27:12.084622 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc342ffe_374f_4ae8_b458_3f911cb55fa4.slice/crio-c046326bbd754605eda711f7f19ebb30ac89eadbd1db396aacafbb1161322257 WatchSource:0}: Error finding container c046326bbd754605eda711f7f19ebb30ac89eadbd1db396aacafbb1161322257: Status 404 returned error can't find the container with id c046326bbd754605eda711f7f19ebb30ac89eadbd1db396aacafbb1161322257 Jul 08 21:27:12.584233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:12.584201 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" event={"ID":"dc342ffe-374f-4ae8-b458-3f911cb55fa4","Type":"ContainerStarted","Data":"c046326bbd754605eda711f7f19ebb30ac89eadbd1db396aacafbb1161322257"} Jul 08 21:27:35.739360 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:35.739326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" event={"ID":"dc342ffe-374f-4ae8-b458-3f911cb55fa4","Type":"ContainerStarted","Data":"309f68da556d3ededdc9028519d2df50c4ac1cab58b96959f9bcf10483b94172"} Jul 08 21:27:35.754972 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:27:35.754928 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2z6gd" podStartSLOduration=1.786249435 podStartE2EDuration="24.754909516s" totalImagesPullingTime="22.968660081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:35.75324909 +0000 UTC m=+713.663686507" watchObservedRunningTime="2026-07-08 21:27:35.754909516 +0000 UTC m=+713.665346937" Jul 08 21:28:05.230467 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.230433 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:28:05.372077 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.372045 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:28:05.372077 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.372081 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:28:05.372273 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.372168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.374800 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.374777 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 21:28:05.516366 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.516274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ced0dc15-2026-4f86-9278-40b95abfde23-config-file\") pod \"limitador-limitador-54d6d988d5-wcqv4\" (UID: \"ced0dc15-2026-4f86-9278-40b95abfde23\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.516366 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.516331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgdxp\" (UniqueName: \"kubernetes.io/projected/ced0dc15-2026-4f86-9278-40b95abfde23-kube-api-access-dgdxp\") pod \"limitador-limitador-54d6d988d5-wcqv4\" (UID: \"ced0dc15-2026-4f86-9278-40b95abfde23\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.616984 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.616949 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ced0dc15-2026-4f86-9278-40b95abfde23-config-file\") pod \"limitador-limitador-54d6d988d5-wcqv4\" (UID: \"ced0dc15-2026-4f86-9278-40b95abfde23\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.617159 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.617011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgdxp\" (UniqueName: \"kubernetes.io/projected/ced0dc15-2026-4f86-9278-40b95abfde23-kube-api-access-dgdxp\") pod \"limitador-limitador-54d6d988d5-wcqv4\" (UID: \"ced0dc15-2026-4f86-9278-40b95abfde23\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.617554 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.617534 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ced0dc15-2026-4f86-9278-40b95abfde23-config-file\") pod \"limitador-limitador-54d6d988d5-wcqv4\" (UID: \"ced0dc15-2026-4f86-9278-40b95abfde23\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.626958 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.626931 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgdxp\" (UniqueName: \"kubernetes.io/projected/ced0dc15-2026-4f86-9278-40b95abfde23-kube-api-access-dgdxp\") pod \"limitador-limitador-54d6d988d5-wcqv4\" (UID: \"ced0dc15-2026-4f86-9278-40b95abfde23\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.682295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.682260 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:05.818356 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.818327 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9hjlk"] Jul 08 21:28:05.822720 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.822697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:05.825233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.825212 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qd6sm\"" Jul 08 21:28:05.830667 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.830641 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9hjlk"] Jul 08 21:28:05.840837 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:28:05.836696 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced0dc15_2026_4f86_9278_40b95abfde23.slice/crio-308c72e4d23c608ffbbcb5e9c8cc808a1d8dbade63f599b10994e23e54b1641f WatchSource:0}: Error finding container 308c72e4d23c608ffbbcb5e9c8cc808a1d8dbade63f599b10994e23e54b1641f: Status 404 returned error can't find the container with id 308c72e4d23c608ffbbcb5e9c8cc808a1d8dbade63f599b10994e23e54b1641f Jul 08 21:28:05.843028 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.842999 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:28:05.900199 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.900164 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" event={"ID":"ced0dc15-2026-4f86-9278-40b95abfde23","Type":"ContainerStarted","Data":"308c72e4d23c608ffbbcb5e9c8cc808a1d8dbade63f599b10994e23e54b1641f"} Jul 08 21:28:05.920695 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:05.920672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgxfz\" (UniqueName: \"kubernetes.io/projected/51e58a29-bdba-4d12-958b-f166b9c22c8f-kube-api-access-kgxfz\") pod \"authorino-75d556bd77-9hjlk\" (UID: \"51e58a29-bdba-4d12-958b-f166b9c22c8f\") " pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:06.021556 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:06.021518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgxfz\" (UniqueName: \"kubernetes.io/projected/51e58a29-bdba-4d12-958b-f166b9c22c8f-kube-api-access-kgxfz\") pod \"authorino-75d556bd77-9hjlk\" (UID: \"51e58a29-bdba-4d12-958b-f166b9c22c8f\") " pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:06.029675 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:06.029613 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgxfz\" (UniqueName: \"kubernetes.io/projected/51e58a29-bdba-4d12-958b-f166b9c22c8f-kube-api-access-kgxfz\") pod \"authorino-75d556bd77-9hjlk\" (UID: \"51e58a29-bdba-4d12-958b-f166b9c22c8f\") " pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:06.145737 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:06.145704 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:06.292393 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:06.292299 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9hjlk"] Jul 08 21:28:06.294415 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:28:06.294387 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51e58a29_bdba_4d12_958b_f166b9c22c8f.slice/crio-65141afb9e9368b2f4585917548be02ea38712804acecd6096d6631cba826014 WatchSource:0}: Error finding container 65141afb9e9368b2f4585917548be02ea38712804acecd6096d6631cba826014: Status 404 returned error can't find the container with id 65141afb9e9368b2f4585917548be02ea38712804acecd6096d6631cba826014 Jul 08 21:28:06.915006 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:06.914931 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9hjlk" event={"ID":"51e58a29-bdba-4d12-958b-f166b9c22c8f","Type":"ContainerStarted","Data":"65141afb9e9368b2f4585917548be02ea38712804acecd6096d6631cba826014"} Jul 08 21:28:10.941933 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:10.941897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9hjlk" event={"ID":"51e58a29-bdba-4d12-958b-f166b9c22c8f","Type":"ContainerStarted","Data":"555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854"} Jul 08 21:28:10.943537 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:10.943513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" event={"ID":"ced0dc15-2026-4f86-9278-40b95abfde23","Type":"ContainerStarted","Data":"293c6590a0c9b2597a12284ec35852e0a623e412400a33631fdf4d910b417d4b"} Jul 08 21:28:10.943616 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:10.943599 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:10.957934 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:10.957861 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9hjlk" podStartSLOduration=2.219283712 podStartE2EDuration="5.957848846s" totalImagesPullingTime="3.738565134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:10.956645766 +0000 UTC m=+748.867083184" watchObservedRunningTime="2026-07-08 21:28:10.957848846 +0000 UTC m=+748.868286264" Jul 08 21:28:10.975147 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:10.975102 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" podStartSLOduration=1.7819045390000001 podStartE2EDuration="5.975087122s" totalImagesPullingTime="4.193182583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:10.97290141 +0000 UTC m=+748.883338828" watchObservedRunningTime="2026-07-08 21:28:10.975087122 +0000 UTC m=+748.885524538" Jul 08 21:28:21.950584 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:21.950550 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wcqv4" Jul 08 21:28:40.361096 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.361062 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6fd8699f79-snfp7"] Jul 08 21:28:40.363768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.363749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.366170 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.366147 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 21:28:40.372415 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.372396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fd8699f79-snfp7"] Jul 08 21:28:40.412625 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.412599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dvnz\" (UniqueName: \"kubernetes.io/projected/f6e87a07-996e-4ca6-8f76-3c5868601e84-kube-api-access-4dvnz\") pod \"authorino-6fd8699f79-snfp7\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.412741 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.412667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6e87a07-996e-4ca6-8f76-3c5868601e84-tls-cert\") pod \"authorino-6fd8699f79-snfp7\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.513805 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.513774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6e87a07-996e-4ca6-8f76-3c5868601e84-tls-cert\") pod \"authorino-6fd8699f79-snfp7\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.513970 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.513830 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dvnz\" (UniqueName: \"kubernetes.io/projected/f6e87a07-996e-4ca6-8f76-3c5868601e84-kube-api-access-4dvnz\") pod \"authorino-6fd8699f79-snfp7\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.516388 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.516365 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6e87a07-996e-4ca6-8f76-3c5868601e84-tls-cert\") pod \"authorino-6fd8699f79-snfp7\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.521736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.521712 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dvnz\" (UniqueName: \"kubernetes.io/projected/f6e87a07-996e-4ca6-8f76-3c5868601e84-kube-api-access-4dvnz\") pod \"authorino-6fd8699f79-snfp7\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.673989 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.673960 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:28:40.801276 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:40.801252 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fd8699f79-snfp7"] Jul 08 21:28:40.802718 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:28:40.802693 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6e87a07_996e_4ca6_8f76_3c5868601e84.slice/crio-26e094615b8a9721f57e6b31dec28c59a6d878d452b7986b898c52154b05be99 WatchSource:0}: Error finding container 26e094615b8a9721f57e6b31dec28c59a6d878d452b7986b898c52154b05be99: Status 404 returned error can't find the container with id 26e094615b8a9721f57e6b31dec28c59a6d878d452b7986b898c52154b05be99 Jul 08 21:28:41.102632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:41.102551 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd8699f79-snfp7" event={"ID":"f6e87a07-996e-4ca6-8f76-3c5868601e84","Type":"ContainerStarted","Data":"26e094615b8a9721f57e6b31dec28c59a6d878d452b7986b898c52154b05be99"} Jul 08 21:28:42.109480 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.109439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd8699f79-snfp7" event={"ID":"f6e87a07-996e-4ca6-8f76-3c5868601e84","Type":"ContainerStarted","Data":"37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3"} Jul 08 21:28:42.126213 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.126163 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6fd8699f79-snfp7" podStartSLOduration=1.698159433 podStartE2EDuration="2.126148249s" totalImagesPullingTime="427.988816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:42.123896244 +0000 UTC m=+780.034333659" watchObservedRunningTime="2026-07-08 21:28:42.126148249 +0000 UTC m=+780.036585710" Jul 08 21:28:42.147336 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.147303 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9hjlk"] Jul 08 21:28:42.147537 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.147511 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9hjlk" podUID="51e58a29-bdba-4d12-958b-f166b9c22c8f" containerName="authorino" containerID="cri-o://555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854" gracePeriod=30 Jul 08 21:28:42.452853 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.452821 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:42.530820 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.530791 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgxfz\" (UniqueName: \"kubernetes.io/projected/51e58a29-bdba-4d12-958b-f166b9c22c8f-kube-api-access-kgxfz\") pod \"51e58a29-bdba-4d12-958b-f166b9c22c8f\" (UID: \"51e58a29-bdba-4d12-958b-f166b9c22c8f\") " Jul 08 21:28:42.537815 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.533714 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e58a29-bdba-4d12-958b-f166b9c22c8f-kube-api-access-kgxfz" (OuterVolumeSpecName: "kube-api-access-kgxfz") pod "51e58a29-bdba-4d12-958b-f166b9c22c8f" (UID: "51e58a29-bdba-4d12-958b-f166b9c22c8f"). InnerVolumeSpecName "kube-api-access-kgxfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:28:42.632394 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:42.632365 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgxfz\" (UniqueName: \"kubernetes.io/projected/51e58a29-bdba-4d12-958b-f166b9c22c8f-kube-api-access-kgxfz\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:28:43.116461 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.116426 2578 generic.go:356] "Generic (PLEG): container finished" podID="51e58a29-bdba-4d12-958b-f166b9c22c8f" containerID="555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854" exitCode=0 Jul 08 21:28:43.116836 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.116491 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9hjlk" Jul 08 21:28:43.116836 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.116504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9hjlk" event={"ID":"51e58a29-bdba-4d12-958b-f166b9c22c8f","Type":"ContainerDied","Data":"555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854"} Jul 08 21:28:43.116836 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.116546 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9hjlk" event={"ID":"51e58a29-bdba-4d12-958b-f166b9c22c8f","Type":"ContainerDied","Data":"65141afb9e9368b2f4585917548be02ea38712804acecd6096d6631cba826014"} Jul 08 21:28:43.116836 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.116565 2578 scope.go:117] "RemoveContainer" containerID="555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854" Jul 08 21:28:43.125313 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.125285 2578 scope.go:117] "RemoveContainer" containerID="555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854" Jul 08 21:28:43.125562 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:28:43.125542 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854\": container with ID starting with 555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854 not found: ID does not exist" containerID="555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854" Jul 08 21:28:43.125627 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.125575 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854"} err="failed to get container status \"555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854\": rpc error: code = NotFound desc = could not find container \"555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854\": container with ID starting with 555f49759bdbd1cfc87b0a0a1ce9cc0d3d6bbe45e64cf376d7cec0572e502854 not found: ID does not exist" Jul 08 21:28:43.131879 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.131843 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9hjlk"] Jul 08 21:28:43.135382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:43.135352 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9hjlk"] Jul 08 21:28:44.616427 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:28:44.616398 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e58a29-bdba-4d12-958b-f166b9c22c8f" path="/var/lib/kubelet/pods/51e58a29-bdba-4d12-958b-f166b9c22c8f/volumes" Jul 08 21:29:20.364674 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:29:20.364602 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:29:37.839890 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:29:37.839837 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:29:44.922470 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:29:44.922436 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:29:54.531564 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:29:54.531499 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:30:00.123123 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:00.123087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:30:12.934781 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:12.934739 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:30:17.620406 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.620372 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm"] Jul 08 21:30:17.620751 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.620733 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e58a29-bdba-4d12-958b-f166b9c22c8f" containerName="authorino" Jul 08 21:30:17.620751 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.620745 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e58a29-bdba-4d12-958b-f166b9c22c8f" containerName="authorino" Jul 08 21:30:17.620826 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.620819 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e58a29-bdba-4d12-958b-f166b9c22c8f" containerName="authorino" Jul 08 21:30:17.623449 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.623432 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.626832 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.626807 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-74w8c\"" Jul 08 21:30:17.626969 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.626841 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 21:30:17.626969 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.626948 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 21:30:17.626969 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.626964 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 21:30:17.633728 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.633703 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm"] Jul 08 21:30:17.654758 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.654736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.654861 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.654787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579aea95-e632-4a1a-83da-dfc93e1cbe82-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.654861 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.654833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njk9\" (UniqueName: \"kubernetes.io/projected/579aea95-e632-4a1a-83da-dfc93e1cbe82-kube-api-access-8njk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.654953 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.654930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.654994 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.654958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.655031 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.655003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756140 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579aea95-e632-4a1a-83da-dfc93e1cbe82-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8njk9\" (UniqueName: \"kubernetes.io/projected/579aea95-e632-4a1a-83da-dfc93e1cbe82-kube-api-access-8njk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756269 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756486 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756544 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756500 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756699 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.756799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.756701 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.758610 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.758592 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/579aea95-e632-4a1a-83da-dfc93e1cbe82-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.758780 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.758764 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/579aea95-e632-4a1a-83da-dfc93e1cbe82-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.763694 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.763672 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njk9\" (UniqueName: \"kubernetes.io/projected/579aea95-e632-4a1a-83da-dfc93e1cbe82-kube-api-access-8njk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm\" (UID: \"579aea95-e632-4a1a-83da-dfc93e1cbe82\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:17.933986 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:17.933950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:18.065905 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:18.065851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm"] Jul 08 21:30:18.067315 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:30:18.067291 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod579aea95_e632_4a1a_83da_dfc93e1cbe82.slice/crio-ffc208f58179f5e3e58cb0d36e3fa197ced9bc1ae426d305f4fe765a8dffc98c WatchSource:0}: Error finding container ffc208f58179f5e3e58cb0d36e3fa197ced9bc1ae426d305f4fe765a8dffc98c: Status 404 returned error can't find the container with id ffc208f58179f5e3e58cb0d36e3fa197ced9bc1ae426d305f4fe765a8dffc98c Jul 08 21:30:18.645024 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:18.644980 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" event={"ID":"579aea95-e632-4a1a-83da-dfc93e1cbe82","Type":"ContainerStarted","Data":"ffc208f58179f5e3e58cb0d36e3fa197ced9bc1ae426d305f4fe765a8dffc98c"} Jul 08 21:30:19.651351 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:19.651316 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" event={"ID":"579aea95-e632-4a1a-83da-dfc93e1cbe82","Type":"ContainerStarted","Data":"38a90482e045f3113b659b917a45b036c78ff2d7c00b406f15443c4fc61b45f6"} Jul 08 21:30:19.669947 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:19.669896 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" podStartSLOduration=1.535913327 podStartE2EDuration="2.669881534s" totalImagesPullingTime="1.133968207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:30:19.667175364 +0000 UTC m=+877.577612807" watchObservedRunningTime="2026-07-08 21:30:19.669881534 +0000 UTC m=+877.580318945" Jul 08 21:30:20.430010 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:20.429979 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:30:27.934353 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:27.934324 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:27.934741 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:27.934365 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:27.946912 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:27.946859 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:28.710795 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:28.710766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm" Jul 08 21:30:42.573061 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:42.572988 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:30:42.573489 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:30:42.573327 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:31:13.709261 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.709227 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54f54b8558-jqkxk"] Jul 08 21:31:13.711642 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.711627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:13.720406 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.720385 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54f54b8558-jqkxk"] Jul 08 21:31:13.840035 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.839998 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13-tls-cert\") pod \"authorino-54f54b8558-jqkxk\" (UID: \"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13\") " pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:13.840192 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.840054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qww84\" (UniqueName: \"kubernetes.io/projected/3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13-kube-api-access-qww84\") pod \"authorino-54f54b8558-jqkxk\" (UID: \"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13\") " pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:13.941112 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.941080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13-tls-cert\") pod \"authorino-54f54b8558-jqkxk\" (UID: \"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13\") " pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:13.941276 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.941124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qww84\" (UniqueName: \"kubernetes.io/projected/3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13-kube-api-access-qww84\") pod \"authorino-54f54b8558-jqkxk\" (UID: \"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13\") " pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:13.943675 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.943652 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13-tls-cert\") pod \"authorino-54f54b8558-jqkxk\" (UID: \"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13\") " pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:13.948261 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:13.948240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qww84\" (UniqueName: \"kubernetes.io/projected/3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13-kube-api-access-qww84\") pod \"authorino-54f54b8558-jqkxk\" (UID: \"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13\") " pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:14.021028 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.020957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54f54b8558-jqkxk" Jul 08 21:31:14.149368 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.149341 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54f54b8558-jqkxk"] Jul 08 21:31:14.151629 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:31:14.151604 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ac24fa7_ea8a_4c99_95a2_1fab5e1e3c13.slice/crio-fd5443ef4ade2a412c1d5fd42834d72d4c47bd57f2ad3a136ffdb4001df3c356 WatchSource:0}: Error finding container fd5443ef4ade2a412c1d5fd42834d72d4c47bd57f2ad3a136ffdb4001df3c356: Status 404 returned error can't find the container with id fd5443ef4ade2a412c1d5fd42834d72d4c47bd57f2ad3a136ffdb4001df3c356 Jul 08 21:31:14.946570 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.946477 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f54b8558-jqkxk" event={"ID":"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13","Type":"ContainerStarted","Data":"55d7dc7d9e7dd0677ab87e40d551350c3b2f26f8ae5679bdb97b08cd52e423c0"} Jul 08 21:31:14.946570 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.946526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54f54b8558-jqkxk" event={"ID":"3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13","Type":"ContainerStarted","Data":"fd5443ef4ade2a412c1d5fd42834d72d4c47bd57f2ad3a136ffdb4001df3c356"} Jul 08 21:31:14.963955 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.963908 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54f54b8558-jqkxk" podStartSLOduration=1.463528899 podStartE2EDuration="1.963890739s" totalImagesPullingTime="500.36184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:31:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:31:14.962058081 +0000 UTC m=+932.872495499" watchObservedRunningTime="2026-07-08 21:31:14.963890739 +0000 UTC m=+932.874328148" Jul 08 21:31:15.000006 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.999070 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6fd8699f79-snfp7"] Jul 08 21:31:15.000006 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:14.999897 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6fd8699f79-snfp7" podUID="f6e87a07-996e-4ca6-8f76-3c5868601e84" containerName="authorino" containerID="cri-o://37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3" gracePeriod=30 Jul 08 21:31:15.267442 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.267416 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:31:15.355598 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.355565 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dvnz\" (UniqueName: \"kubernetes.io/projected/f6e87a07-996e-4ca6-8f76-3c5868601e84-kube-api-access-4dvnz\") pod \"f6e87a07-996e-4ca6-8f76-3c5868601e84\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " Jul 08 21:31:15.355760 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.355627 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6e87a07-996e-4ca6-8f76-3c5868601e84-tls-cert\") pod \"f6e87a07-996e-4ca6-8f76-3c5868601e84\" (UID: \"f6e87a07-996e-4ca6-8f76-3c5868601e84\") " Jul 08 21:31:15.357783 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.357748 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e87a07-996e-4ca6-8f76-3c5868601e84-kube-api-access-4dvnz" (OuterVolumeSpecName: "kube-api-access-4dvnz") pod "f6e87a07-996e-4ca6-8f76-3c5868601e84" (UID: "f6e87a07-996e-4ca6-8f76-3c5868601e84"). InnerVolumeSpecName "kube-api-access-4dvnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:31:15.366487 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.366456 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e87a07-996e-4ca6-8f76-3c5868601e84-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f6e87a07-996e-4ca6-8f76-3c5868601e84" (UID: "f6e87a07-996e-4ca6-8f76-3c5868601e84"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:31:15.456912 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.456852 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dvnz\" (UniqueName: \"kubernetes.io/projected/f6e87a07-996e-4ca6-8f76-3c5868601e84-kube-api-access-4dvnz\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:31:15.456912 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.456909 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6e87a07-996e-4ca6-8f76-3c5868601e84-tls-cert\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:31:15.960631 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.960585 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6e87a07-996e-4ca6-8f76-3c5868601e84" containerID="37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3" exitCode=0 Jul 08 21:31:15.960999 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.960760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd8699f79-snfp7" event={"ID":"f6e87a07-996e-4ca6-8f76-3c5868601e84","Type":"ContainerDied","Data":"37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3"} Jul 08 21:31:15.960999 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.960834 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fd8699f79-snfp7" Jul 08 21:31:15.960999 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.960831 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fd8699f79-snfp7" event={"ID":"f6e87a07-996e-4ca6-8f76-3c5868601e84","Type":"ContainerDied","Data":"26e094615b8a9721f57e6b31dec28c59a6d878d452b7986b898c52154b05be99"} Jul 08 21:31:15.960999 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.960885 2578 scope.go:117] "RemoveContainer" containerID="37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3" Jul 08 21:31:15.970568 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.970549 2578 scope.go:117] "RemoveContainer" containerID="37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3" Jul 08 21:31:15.970833 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:31:15.970814 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3\": container with ID starting with 37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3 not found: ID does not exist" containerID="37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3" Jul 08 21:31:15.970911 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.970841 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3"} err="failed to get container status \"37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3\": rpc error: code = NotFound desc = could not find container \"37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3\": container with ID starting with 37682392be3ce00a63a088b34635d1ebd58e8a60fe2115060a51d50f06b028a3 not found: ID does not exist" Jul 08 21:31:15.984929 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.984903 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6fd8699f79-snfp7"] Jul 08 21:31:15.988808 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:15.988777 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6fd8699f79-snfp7"] Jul 08 21:31:16.616100 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:16.616066 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e87a07-996e-4ca6-8f76-3c5868601e84" path="/var/lib/kubelet/pods/f6e87a07-996e-4ca6-8f76-3c5868601e84/volumes" Jul 08 21:31:44.223316 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:44.223280 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:31:54.623298 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:31:54.623261 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:32:05.222350 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:32:05.222276 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:32:15.531001 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:32:15.530962 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:32:25.121810 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:32:25.121773 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:32:53.626776 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:32:53.626742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:33:07.620623 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:33:07.620585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:33:40.966844 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:33:40.966765 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:33:50.527610 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:33:50.527578 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:33:59.730474 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:33:59.730438 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:34:11.319304 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:34:11.319266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:34:21.930459 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:34:21.930426 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:34:31.669683 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:34:31.669648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:34:42.923013 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:34:42.922981 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:34:52.926908 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:34:52.926877 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:35:18.226135 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:35:18.226056 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:35:30.024933 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:35:30.024892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:35:42.614787 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:35:42.614751 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:35:42.623991 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:35:42.623960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:35:54.124150 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:35:54.124111 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:36:05.325274 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:36:05.325234 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:36:18.931324 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:36:18.931288 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:36:29.826724 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:36:29.826691 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:36:39.226847 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:36:39.226763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:36:48.630736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:36:48.630696 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:36:56.824108 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:36:56.824072 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:38:47.238480 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:38:47.238422 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:38:55.632465 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:38:55.632429 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:39:08.734209 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:39:08.734169 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:39:20.333740 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:39:20.333704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:39:32.532063 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:39:32.532027 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:39:42.127887 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:39:42.127839 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:39:51.536977 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:39:51.536941 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:40:01.628835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:01.628799 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:40:12.831719 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:12.831687 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:40:22.025779 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:22.025741 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:40:32.925342 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:32.925309 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:40:42.661065 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:42.661039 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:40:42.664093 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:42.664074 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:40:54.623995 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:40:54.623961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:41:03.737831 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:41:03.737751 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:41:13.332694 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:41:13.332659 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:41:25.144541 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:41:25.144502 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:41:41.222745 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:41:41.222706 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:41:50.927774 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:41:50.927740 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:42:00.227754 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:42:00.227717 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:42:11.138731 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:42:11.138693 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:42:21.027808 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:42:21.027773 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:42:31.333036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:42:31.333004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:42:43.232463 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:42:43.232370 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:42:52.646983 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:42:52.646949 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:43:02.825303 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:43:02.825268 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:43:15.426361 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:43:15.426319 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:43:24.421417 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:43:24.421376 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:43:31.430920 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:43:31.430886 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:43:48.330034 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:43:48.329989 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:43:59.825320 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:43:59.825289 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:44:13.231330 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:44:13.231251 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:44:26.523206 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:44:26.523172 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:44:38.427305 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:44:38.427269 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:44:51.331679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:44:51.331644 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:45:00.134992 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.134960 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg"] Jul 08 21:45:00.135375 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.135341 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e87a07-996e-4ca6-8f76-3c5868601e84" containerName="authorino" Jul 08 21:45:00.135375 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.135352 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e87a07-996e-4ca6-8f76-3c5868601e84" containerName="authorino" Jul 08 21:45:00.135447 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.135435 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e87a07-996e-4ca6-8f76-3c5868601e84" containerName="authorino" Jul 08 21:45:00.138581 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.138561 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:45:00.141213 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.141192 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 21:45:00.141314 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.141192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xdh9f\"" Jul 08 21:45:00.142192 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.142177 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 21:45:00.146740 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.146718 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg"] Jul 08 21:45:00.258514 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.258482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnplf\" (UniqueName: \"kubernetes.io/projected/58244a21-17d5-47dd-bf7c-435246768e5f-kube-api-access-bnplf\") pod \"maas-api-key-cleanup-29725785-q4zsg\" (UID: \"58244a21-17d5-47dd-bf7c-435246768e5f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:45:00.359664 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.359639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnplf\" (UniqueName: \"kubernetes.io/projected/58244a21-17d5-47dd-bf7c-435246768e5f-kube-api-access-bnplf\") pod \"maas-api-key-cleanup-29725785-q4zsg\" (UID: \"58244a21-17d5-47dd-bf7c-435246768e5f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:45:00.368425 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.368393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnplf\" (UniqueName: \"kubernetes.io/projected/58244a21-17d5-47dd-bf7c-435246768e5f-kube-api-access-bnplf\") pod \"maas-api-key-cleanup-29725785-q4zsg\" (UID: \"58244a21-17d5-47dd-bf7c-435246768e5f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:45:00.427907 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.427821 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:45:00.450429 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.450401 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:45:00.595265 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.595238 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg"] Jul 08 21:45:00.596982 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:45:00.596952 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58244a21_17d5_47dd_bf7c_435246768e5f.slice/crio-7481d0b87db620944712ea44d711ad49b322a8e27483aae210ef6fbc6970da8f WatchSource:0}: Error finding container 7481d0b87db620944712ea44d711ad49b322a8e27483aae210ef6fbc6970da8f: Status 404 returned error can't find the container with id 7481d0b87db620944712ea44d711ad49b322a8e27483aae210ef6fbc6970da8f Jul 08 21:45:00.598835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:00.598822 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:45:01.497702 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:01.497668 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" event={"ID":"58244a21-17d5-47dd-bf7c-435246768e5f","Type":"ContainerStarted","Data":"7481d0b87db620944712ea44d711ad49b322a8e27483aae210ef6fbc6970da8f"} Jul 08 21:45:03.510721 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:03.510681 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" event={"ID":"58244a21-17d5-47dd-bf7c-435246768e5f","Type":"ContainerStarted","Data":"718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e"} Jul 08 21:45:03.530480 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:03.530432 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" podStartSLOduration=1.225738414 podStartE2EDuration="3.530417944s" totalImagesPullingTime="2.30467953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:45:03.528327217 +0000 UTC m=+1761.438764636" watchObservedRunningTime="2026-07-08 21:45:03.530417944 +0000 UTC m=+1761.440855358" Jul 08 21:45:12.629004 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:12.628969 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:45:42.699254 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:42.699231 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:45:42.705586 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:45:42.705565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:46:00.011127 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:00.011089 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg"] Jul 08 21:46:00.011615 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:00.011281 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" podUID="58244a21-17d5-47dd-bf7c-435246768e5f" containerName="cleanup" containerID="cri-o://718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e" gracePeriod=30 Jul 08 21:46:30.201741 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.201717 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:46:30.238476 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.238449 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnplf\" (UniqueName: \"kubernetes.io/projected/58244a21-17d5-47dd-bf7c-435246768e5f-kube-api-access-bnplf\") pod \"58244a21-17d5-47dd-bf7c-435246768e5f\" (UID: \"58244a21-17d5-47dd-bf7c-435246768e5f\") " Jul 08 21:46:30.240651 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.240629 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58244a21-17d5-47dd-bf7c-435246768e5f-kube-api-access-bnplf" (OuterVolumeSpecName: "kube-api-access-bnplf") pod "58244a21-17d5-47dd-bf7c-435246768e5f" (UID: "58244a21-17d5-47dd-bf7c-435246768e5f"). InnerVolumeSpecName "kube-api-access-bnplf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:46:30.340033 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.339954 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnplf\" (UniqueName: \"kubernetes.io/projected/58244a21-17d5-47dd-bf7c-435246768e5f-kube-api-access-bnplf\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:46:30.992561 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.992529 2578 generic.go:356] "Generic (PLEG): container finished" podID="58244a21-17d5-47dd-bf7c-435246768e5f" containerID="718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e" exitCode=137 Jul 08 21:46:30.992736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.992576 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" Jul 08 21:46:30.992736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.992618 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" event={"ID":"58244a21-17d5-47dd-bf7c-435246768e5f","Type":"ContainerDied","Data":"718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e"} Jul 08 21:46:30.992736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.992673 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg" event={"ID":"58244a21-17d5-47dd-bf7c-435246768e5f","Type":"ContainerDied","Data":"7481d0b87db620944712ea44d711ad49b322a8e27483aae210ef6fbc6970da8f"} Jul 08 21:46:30.992736 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:30.992701 2578 scope.go:117] "RemoveContainer" containerID="718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e" Jul 08 21:46:31.001528 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:31.001501 2578 scope.go:117] "RemoveContainer" containerID="718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e" Jul 08 21:46:31.001780 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:46:31.001762 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e\": container with ID starting with 718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e not found: ID does not exist" containerID="718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e" Jul 08 21:46:31.001847 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:31.001787 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e"} err="failed to get container status \"718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e\": rpc error: code = NotFound desc = could not find container \"718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e\": container with ID starting with 718407a8e2fb588e4fb7950d32a2a1bc1afe6701003c66c276f5118ac955e15e not found: ID does not exist" Jul 08 21:46:31.009645 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:31.009620 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg"] Jul 08 21:46:31.015652 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:31.015625 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725785-q4zsg"] Jul 08 21:46:32.616268 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:32.616235 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58244a21-17d5-47dd-bf7c-435246768e5f" path="/var/lib/kubelet/pods/58244a21-17d5-47dd-bf7c-435246768e5f/volumes" Jul 08 21:46:49.134688 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:49.134650 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:46:58.527546 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:46:58.527508 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:47:09.532739 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:09.532656 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:47:20.332608 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:20.332574 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:47:50.220721 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:50.220690 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:47:58.928281 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.928247 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f"] Jul 08 21:47:58.928843 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.928827 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58244a21-17d5-47dd-bf7c-435246768e5f" containerName="cleanup" Jul 08 21:47:58.928913 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.928847 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="58244a21-17d5-47dd-bf7c-435246768e5f" containerName="cleanup" Jul 08 21:47:58.928974 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.928963 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="58244a21-17d5-47dd-bf7c-435246768e5f" containerName="cleanup" Jul 08 21:47:58.932562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.932543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:58.935173 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.935153 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9b10366\"/\"openshift-service-ca.crt\"" Jul 08 21:47:58.936329 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.936288 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9b10366\"/\"kube-root-ca.crt\"" Jul 08 21:47:58.936329 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.936288 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9b10366\"/\"e2e-lifecycle-model-e9b10366-kserve-self-signed-certs\"" Jul 08 21:47:58.936604 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.936589 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9b10366\"/\"default-dockercfg-c2k2c\"" Jul 08 21:47:58.939184 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:58.939162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f"] Jul 08 21:47:59.039211 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.039178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-dshm\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.039381 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.039221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-model-cache\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.039381 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.039254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2m5n\" (UniqueName: \"kubernetes.io/projected/d31ace5d-d3e9-4641-ad2a-b484b0a50695-kube-api-access-s2m5n\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.039381 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.039278 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tmp-dir\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.039511 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.039382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-home\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.039511 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.039407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tls-certs\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140353 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tmp-dir\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140486 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-home\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140486 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tls-certs\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140486 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-dshm\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140486 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-model-cache\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140699 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2m5n\" (UniqueName: \"kubernetes.io/projected/d31ace5d-d3e9-4641-ad2a-b484b0a50695-kube-api-access-s2m5n\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140791 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tmp-dir\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140855 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140808 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-home\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.140941 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.140897 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-model-cache\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.142821 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.142798 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-dshm\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.143151 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.143134 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tls-certs\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.148887 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.148854 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2m5n\" (UniqueName: \"kubernetes.io/projected/d31ace5d-d3e9-4641-ad2a-b484b0a50695-kube-api-access-s2m5n\") pod \"e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.243859 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.243785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:47:59.375282 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.375256 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f"] Jul 08 21:47:59.376691 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:47:59.376665 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd31ace5d_d3e9_4641_ad2a_b484b0a50695.slice/crio-a0fc279ef8551fef2fad3895af84968e1c0fd5e1afc08c1c0d51c8fd3285dcea WatchSource:0}: Error finding container a0fc279ef8551fef2fad3895af84968e1c0fd5e1afc08c1c0d51c8fd3285dcea: Status 404 returned error can't find the container with id a0fc279ef8551fef2fad3895af84968e1c0fd5e1afc08c1c0d51c8fd3285dcea Jul 08 21:47:59.475228 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.475189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" event={"ID":"d31ace5d-d3e9-4641-ad2a-b484b0a50695","Type":"ContainerStarted","Data":"308e4f499d467e96992366f2c5483d4664eee55eb0176e3b3b18638c69a5ed32"} Jul 08 21:47:59.475402 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.475238 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" event={"ID":"d31ace5d-d3e9-4641-ad2a-b484b0a50695","Type":"ContainerStarted","Data":"a0fc279ef8551fef2fad3895af84968e1c0fd5e1afc08c1c0d51c8fd3285dcea"} Jul 08 21:47:59.515188 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:47:59.515097 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" podStartSLOduration=1.515079426 podStartE2EDuration="1.515079426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:47:59.514801906 +0000 UTC m=+1937.425239324" watchObservedRunningTime="2026-07-08 21:47:59.515079426 +0000 UTC m=+1937.425516843" Jul 08 21:48:05.324394 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:05.324357 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:48:09.244031 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:09.243992 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:48:09.244407 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:09.244065 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:48:09.256382 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:09.256355 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:48:09.541372 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:09.541293 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:48:14.967253 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:14.967216 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:48:26.928010 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:26.927968 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:48:32.466481 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:32.466452 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f"] Jul 08 21:48:36.733224 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:36.733191 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:48:47.486666 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.486605 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" podUID="d31ace5d-d3e9-4641-ad2a-b484b0a50695" containerName="main" containerID="cri-o://308e4f499d467e96992366f2c5483d4664eee55eb0176e3b3b18638c69a5ed32" gracePeriod=45 Jul 08 21:48:47.734711 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.734455 2578 generic.go:356] "Generic (PLEG): container finished" podID="d31ace5d-d3e9-4641-ad2a-b484b0a50695" containerID="308e4f499d467e96992366f2c5483d4664eee55eb0176e3b3b18638c69a5ed32" exitCode=0 Jul 08 21:48:47.734711 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.734603 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" event={"ID":"d31ace5d-d3e9-4641-ad2a-b484b0a50695","Type":"ContainerDied","Data":"308e4f499d467e96992366f2c5483d4664eee55eb0176e3b3b18638c69a5ed32"} Jul 08 21:48:47.734711 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.734637 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" event={"ID":"d31ace5d-d3e9-4641-ad2a-b484b0a50695","Type":"ContainerDied","Data":"a0fc279ef8551fef2fad3895af84968e1c0fd5e1afc08c1c0d51c8fd3285dcea"} Jul 08 21:48:47.734711 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.734652 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0fc279ef8551fef2fad3895af84968e1c0fd5e1afc08c1c0d51c8fd3285dcea" Jul 08 21:48:47.737552 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.737504 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:48:47.835101 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835065 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-dshm\") pod \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " Jul 08 21:48:47.835295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835109 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tmp-dir\") pod \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " Jul 08 21:48:47.835295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835141 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-model-cache\") pod \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " Jul 08 21:48:47.835295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835197 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-home\") pod \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " Jul 08 21:48:47.835295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835237 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2m5n\" (UniqueName: \"kubernetes.io/projected/d31ace5d-d3e9-4641-ad2a-b484b0a50695-kube-api-access-s2m5n\") pod \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " Jul 08 21:48:47.835295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835270 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tls-certs\") pod \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\" (UID: \"d31ace5d-d3e9-4641-ad2a-b484b0a50695\") " Jul 08 21:48:47.835545 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835389 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d31ace5d-d3e9-4641-ad2a-b484b0a50695" (UID: "d31ace5d-d3e9-4641-ad2a-b484b0a50695"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:48:47.835545 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835432 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-model-cache" (OuterVolumeSpecName: "model-cache") pod "d31ace5d-d3e9-4641-ad2a-b484b0a50695" (UID: "d31ace5d-d3e9-4641-ad2a-b484b0a50695"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:48:47.835545 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835447 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-home" (OuterVolumeSpecName: "home") pod "d31ace5d-d3e9-4641-ad2a-b484b0a50695" (UID: "d31ace5d-d3e9-4641-ad2a-b484b0a50695"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:48:47.835663 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835583 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tmp-dir\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:48:47.835663 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835605 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-model-cache\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:48:47.835663 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.835616 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-home\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:48:47.837426 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.837404 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-dshm" (OuterVolumeSpecName: "dshm") pod "d31ace5d-d3e9-4641-ad2a-b484b0a50695" (UID: "d31ace5d-d3e9-4641-ad2a-b484b0a50695"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:48:47.837534 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.837492 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31ace5d-d3e9-4641-ad2a-b484b0a50695-kube-api-access-s2m5n" (OuterVolumeSpecName: "kube-api-access-s2m5n") pod "d31ace5d-d3e9-4641-ad2a-b484b0a50695" (UID: "d31ace5d-d3e9-4641-ad2a-b484b0a50695"). InnerVolumeSpecName "kube-api-access-s2m5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:48:47.837534 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.837506 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d31ace5d-d3e9-4641-ad2a-b484b0a50695" (UID: "d31ace5d-d3e9-4641-ad2a-b484b0a50695"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:48:47.936385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.936361 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31ace5d-d3e9-4641-ad2a-b484b0a50695-dshm\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:48:47.936385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.936385 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2m5n\" (UniqueName: \"kubernetes.io/projected/d31ace5d-d3e9-4641-ad2a-b484b0a50695-kube-api-access-s2m5n\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:48:47.936385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:47.936396 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31ace5d-d3e9-4641-ad2a-b484b0a50695-tls-certs\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:48:48.739731 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:48.739701 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f" Jul 08 21:48:48.756086 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:48.756060 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f"] Jul 08 21:48:48.759586 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:48.759564 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e9b10366/e2e-lifecycle-model-e9b10366-kserve-6dfb5f7dc-j769f"] Jul 08 21:48:50.615471 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:48:50.615442 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d31ace5d-d3e9-4641-ad2a-b484b0a50695" path="/var/lib/kubelet/pods/d31ace5d-d3e9-4641-ad2a-b484b0a50695/volumes" Jul 08 21:49:12.430485 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:49:12.430452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:49:21.926128 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:49:21.926095 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:49:32.632026 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:49:32.631989 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:50:21.632667 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:21.632585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:50:27.425586 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.425550 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2"] Jul 08 21:50:27.426218 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.426188 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d31ace5d-d3e9-4641-ad2a-b484b0a50695" containerName="main" Jul 08 21:50:27.426218 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.426207 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d31ace5d-d3e9-4641-ad2a-b484b0a50695" containerName="main" Jul 08 21:50:27.426344 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.426328 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d31ace5d-d3e9-4641-ad2a-b484b0a50695" containerName="main" Jul 08 21:50:27.429499 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.429479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.432300 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.432276 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-66014e\"/\"kube-root-ca.crt\"" Jul 08 21:50:27.433285 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.433261 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-66014e\"/\"openshift-service-ca.crt\"" Jul 08 21:50:27.433285 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.433277 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-66014e\"/\"default-dockercfg-h67g9\"" Jul 08 21:50:27.433457 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.433332 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-66014e\"/\"test-model-66014e-kserve-self-signed-certs\"" Jul 08 21:50:27.441015 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.440991 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2"] Jul 08 21:50:27.496809 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.496777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccrnv\" (UniqueName: \"kubernetes.io/projected/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-kube-api-access-ccrnv\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.497016 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.496828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-home\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.497016 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.496949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-dshm\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.497016 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.496987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tmp-dir\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.497159 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.497086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tls-certs\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.497159 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.497132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-model-cache\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.597550 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.597518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tls-certs\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.597710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.597577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-model-cache\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.597710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.597608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrnv\" (UniqueName: \"kubernetes.io/projected/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-kube-api-access-ccrnv\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.597710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.597653 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-home\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.597710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.597680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-dshm\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.597710 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.597705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tmp-dir\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.598073 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.598053 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-model-cache\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.598144 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.598099 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-home\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.598194 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.598139 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tmp-dir\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.600024 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.600004 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-dshm\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.600343 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.600322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tls-certs\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.606507 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.606484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccrnv\" (UniqueName: \"kubernetes.io/projected/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-kube-api-access-ccrnv\") pod \"test-model-66014e-kserve-7d565d49fc-zvhx2\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.741096 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.741011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:27.876666 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:27.876639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2"] Jul 08 21:50:27.877948 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:50:27.877916 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod161a0b79_71c8_47c7_95ed_122d9bb1fbd8.slice/crio-09a10f44539c65ca2b554063e7e192174e7dcbf682a4b342d3dc61241865be2c WatchSource:0}: Error finding container 09a10f44539c65ca2b554063e7e192174e7dcbf682a4b342d3dc61241865be2c: Status 404 returned error can't find the container with id 09a10f44539c65ca2b554063e7e192174e7dcbf682a4b342d3dc61241865be2c Jul 08 21:50:28.275614 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:28.275582 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" event={"ID":"161a0b79-71c8-47c7-95ed-122d9bb1fbd8","Type":"ContainerStarted","Data":"4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e"} Jul 08 21:50:28.275614 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:28.275620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" event={"ID":"161a0b79-71c8-47c7-95ed-122d9bb1fbd8","Type":"ContainerStarted","Data":"09a10f44539c65ca2b554063e7e192174e7dcbf682a4b342d3dc61241865be2c"} Jul 08 21:50:28.294359 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:28.294315 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" podStartSLOduration=1.294300928 podStartE2EDuration="1.294300928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:50:28.292050639 +0000 UTC m=+2086.202488049" watchObservedRunningTime="2026-07-08 21:50:28.294300928 +0000 UTC m=+2086.204738363" Jul 08 21:50:37.741634 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:37.741603 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:37.742146 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:37.741655 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:37.753832 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:37.753810 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:38.342932 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:38.342903 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:50:39.422659 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:39.422618 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:50:42.739738 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:42.739713 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:50:42.753752 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:42.753730 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:50:47.930329 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.930294 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2"] Jul 08 21:50:47.934435 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.934412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:47.939560 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.938160 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9b1f52\"/\"default-dockercfg-qhfzv\"" Jul 08 21:50:47.939560 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.938380 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9b1f52\"/\"openshift-service-ca.crt\"" Jul 08 21:50:47.939560 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.938566 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9b1f52\"/\"test-model-9b1f52-kserve-self-signed-certs\"" Jul 08 21:50:47.940075 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.940052 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9b1f52\"/\"kube-root-ca.crt\"" Jul 08 21:50:47.942356 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.942334 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2"] Jul 08 21:50:47.966133 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.966106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-dshm\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:47.966243 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.966135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fcc59c-95e0-4dce-9598-0bae15408208-tls-certs\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:47.966243 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.966176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-home\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:47.966243 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.966220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-model-cache\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:47.966367 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.966250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w47hs\" (UniqueName: \"kubernetes.io/projected/35fcc59c-95e0-4dce-9598-0bae15408208-kube-api-access-w47hs\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:47.966367 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:47.966274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-tmp-dir\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.067632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.067598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-dshm\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.067632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.067631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fcc59c-95e0-4dce-9598-0bae15408208-tls-certs\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.067835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.067660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-home\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.067835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.067683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-model-cache\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.067989 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.067862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w47hs\" (UniqueName: \"kubernetes.io/projected/35fcc59c-95e0-4dce-9598-0bae15408208-kube-api-access-w47hs\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.067989 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.067940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-tmp-dir\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.068144 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.068124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-model-cache\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.068223 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.068145 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-home\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.068337 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.068319 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-tmp-dir\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.070014 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.069999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-dshm\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.070322 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.070303 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fcc59c-95e0-4dce-9598-0bae15408208-tls-certs\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.079708 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.079683 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w47hs\" (UniqueName: \"kubernetes.io/projected/35fcc59c-95e0-4dce-9598-0bae15408208-kube-api-access-w47hs\") pod \"test-model-9b1f52-kserve-65766bc5ff-w9fq2\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.247680 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.247594 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:48.382910 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.382883 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2"] Jul 08 21:50:48.384435 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:50:48.384408 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35fcc59c_95e0_4dce_9598_0bae15408208.slice/crio-39026a0caccb5b252abce7e3f2e2ddd0932b520f48bd2ebd895f82a5a7a8f926 WatchSource:0}: Error finding container 39026a0caccb5b252abce7e3f2e2ddd0932b520f48bd2ebd895f82a5a7a8f926: Status 404 returned error can't find the container with id 39026a0caccb5b252abce7e3f2e2ddd0932b520f48bd2ebd895f82a5a7a8f926 Jul 08 21:50:48.393358 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:48.393324 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" event={"ID":"35fcc59c-95e0-4dce-9598-0bae15408208","Type":"ContainerStarted","Data":"39026a0caccb5b252abce7e3f2e2ddd0932b520f48bd2ebd895f82a5a7a8f926"} Jul 08 21:50:49.403241 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:49.403203 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" event={"ID":"35fcc59c-95e0-4dce-9598-0bae15408208","Type":"ContainerStarted","Data":"6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11"} Jul 08 21:50:49.420514 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:49.420463 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" podStartSLOduration=2.420448062 podStartE2EDuration="2.420448062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:50:49.417476916 +0000 UTC m=+2107.327914332" watchObservedRunningTime="2026-07-08 21:50:49.420448062 +0000 UTC m=+2107.330885478" Jul 08 21:50:51.033178 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:51.033138 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:50:58.248372 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:58.248327 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:58.248824 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:58.248386 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:58.261137 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:58.261114 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:50:58.464572 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:50:58.464533 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:51:04.829897 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:04.829850 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:51:08.732504 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:08.732473 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2"] Jul 08 21:51:09.628452 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:09.628424 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2"] Jul 08 21:51:22.021945 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:22.021910 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:51:23.753502 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:23.753465 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" podUID="161a0b79-71c8-47c7-95ed-122d9bb1fbd8" containerName="main" containerID="cri-o://4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e" gracePeriod=45 Jul 08 21:51:24.005272 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.005215 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:51:24.102275 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102245 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tls-certs\") pod \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " Jul 08 21:51:24.102431 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102295 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-dshm\") pod \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " Jul 08 21:51:24.102431 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102319 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tmp-dir\") pod \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " Jul 08 21:51:24.102431 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102354 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-model-cache\") pod \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " Jul 08 21:51:24.102431 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102395 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-home\") pod \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " Jul 08 21:51:24.102633 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102439 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccrnv\" (UniqueName: \"kubernetes.io/projected/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-kube-api-access-ccrnv\") pod \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\" (UID: \"161a0b79-71c8-47c7-95ed-122d9bb1fbd8\") " Jul 08 21:51:24.102693 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102659 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "161a0b79-71c8-47c7-95ed-122d9bb1fbd8" (UID: "161a0b79-71c8-47c7-95ed-122d9bb1fbd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:24.102693 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102668 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "161a0b79-71c8-47c7-95ed-122d9bb1fbd8" (UID: "161a0b79-71c8-47c7-95ed-122d9bb1fbd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:24.102801 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102754 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-home" (OuterVolumeSpecName: "home") pod "161a0b79-71c8-47c7-95ed-122d9bb1fbd8" (UID: "161a0b79-71c8-47c7-95ed-122d9bb1fbd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:24.102801 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102785 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tmp-dir\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:24.102902 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.102806 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-model-cache\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:24.104820 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.104795 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-kube-api-access-ccrnv" (OuterVolumeSpecName: "kube-api-access-ccrnv") pod "161a0b79-71c8-47c7-95ed-122d9bb1fbd8" (UID: "161a0b79-71c8-47c7-95ed-122d9bb1fbd8"). InnerVolumeSpecName "kube-api-access-ccrnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:51:24.105048 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.105023 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "161a0b79-71c8-47c7-95ed-122d9bb1fbd8" (UID: "161a0b79-71c8-47c7-95ed-122d9bb1fbd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:51:24.105048 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.105034 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-dshm" (OuterVolumeSpecName: "dshm") pod "161a0b79-71c8-47c7-95ed-122d9bb1fbd8" (UID: "161a0b79-71c8-47c7-95ed-122d9bb1fbd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:24.203216 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.203189 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-home\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:24.203216 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.203212 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccrnv\" (UniqueName: \"kubernetes.io/projected/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-kube-api-access-ccrnv\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:24.203385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.203223 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-tls-certs\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:24.203385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.203233 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161a0b79-71c8-47c7-95ed-122d9bb1fbd8-dshm\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:24.600913 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.600882 2578 generic.go:356] "Generic (PLEG): container finished" podID="161a0b79-71c8-47c7-95ed-122d9bb1fbd8" containerID="4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e" exitCode=0 Jul 08 21:51:24.601083 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.600921 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" Jul 08 21:51:24.601083 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.600919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" event={"ID":"161a0b79-71c8-47c7-95ed-122d9bb1fbd8","Type":"ContainerDied","Data":"4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e"} Jul 08 21:51:24.601083 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.601039 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2" event={"ID":"161a0b79-71c8-47c7-95ed-122d9bb1fbd8","Type":"ContainerDied","Data":"09a10f44539c65ca2b554063e7e192174e7dcbf682a4b342d3dc61241865be2c"} Jul 08 21:51:24.601083 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.601064 2578 scope.go:117] "RemoveContainer" containerID="4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e" Jul 08 21:51:24.610481 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.610464 2578 scope.go:117] "RemoveContainer" containerID="4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e" Jul 08 21:51:24.610719 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:51:24.610704 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e\": container with ID starting with 4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e not found: ID does not exist" containerID="4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e" Jul 08 21:51:24.610768 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.610727 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e"} err="failed to get container status \"4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e\": rpc error: code = NotFound desc = could not find container \"4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e\": container with ID starting with 4b3321975e86c3869e7c166fd91dc4cbe14ac007aaf96139570787d6269d0f1e not found: ID does not exist" Jul 08 21:51:24.623562 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.623539 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2"] Jul 08 21:51:24.629409 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.629390 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-66014e/test-model-66014e-kserve-7d565d49fc-zvhx2"] Jul 08 21:51:24.648892 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.648847 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" podUID="35fcc59c-95e0-4dce-9598-0bae15408208" containerName="main" containerID="cri-o://6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11" gracePeriod=45 Jul 08 21:51:24.901620 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:24.901600 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:51:25.011795 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.011768 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fcc59c-95e0-4dce-9598-0bae15408208-tls-certs\") pod \"35fcc59c-95e0-4dce-9598-0bae15408208\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " Jul 08 21:51:25.012011 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.011812 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w47hs\" (UniqueName: \"kubernetes.io/projected/35fcc59c-95e0-4dce-9598-0bae15408208-kube-api-access-w47hs\") pod \"35fcc59c-95e0-4dce-9598-0bae15408208\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " Jul 08 21:51:25.012011 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.011893 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-dshm\") pod \"35fcc59c-95e0-4dce-9598-0bae15408208\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " Jul 08 21:51:25.012011 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.011932 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-tmp-dir\") pod \"35fcc59c-95e0-4dce-9598-0bae15408208\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " Jul 08 21:51:25.012011 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.011965 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-model-cache\") pod \"35fcc59c-95e0-4dce-9598-0bae15408208\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " Jul 08 21:51:25.012011 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.011992 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-home\") pod \"35fcc59c-95e0-4dce-9598-0bae15408208\" (UID: \"35fcc59c-95e0-4dce-9598-0bae15408208\") " Jul 08 21:51:25.012295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.012252 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35fcc59c-95e0-4dce-9598-0bae15408208" (UID: "35fcc59c-95e0-4dce-9598-0bae15408208"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:25.012295 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.012263 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-model-cache" (OuterVolumeSpecName: "model-cache") pod "35fcc59c-95e0-4dce-9598-0bae15408208" (UID: "35fcc59c-95e0-4dce-9598-0bae15408208"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:25.012401 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.012298 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-home" (OuterVolumeSpecName: "home") pod "35fcc59c-95e0-4dce-9598-0bae15408208" (UID: "35fcc59c-95e0-4dce-9598-0bae15408208"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:25.014325 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.014305 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35fcc59c-95e0-4dce-9598-0bae15408208-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35fcc59c-95e0-4dce-9598-0bae15408208" (UID: "35fcc59c-95e0-4dce-9598-0bae15408208"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 21:51:25.014396 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.014322 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-dshm" (OuterVolumeSpecName: "dshm") pod "35fcc59c-95e0-4dce-9598-0bae15408208" (UID: "35fcc59c-95e0-4dce-9598-0bae15408208"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:51:25.014396 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.014343 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35fcc59c-95e0-4dce-9598-0bae15408208-kube-api-access-w47hs" (OuterVolumeSpecName: "kube-api-access-w47hs") pod "35fcc59c-95e0-4dce-9598-0bae15408208" (UID: "35fcc59c-95e0-4dce-9598-0bae15408208"). InnerVolumeSpecName "kube-api-access-w47hs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:51:25.112950 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.112917 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fcc59c-95e0-4dce-9598-0bae15408208-tls-certs\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:25.112950 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.112946 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w47hs\" (UniqueName: \"kubernetes.io/projected/35fcc59c-95e0-4dce-9598-0bae15408208-kube-api-access-w47hs\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:25.113111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.112959 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-dshm\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:25.113111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.112972 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-tmp-dir\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:25.113111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.112984 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-model-cache\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:25.113111 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.112994 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fcc59c-95e0-4dce-9598-0bae15408208-home\") on node \"ip-10-0-138-169.ec2.internal\" DevicePath \"\"" Jul 08 21:51:25.608485 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.608449 2578 generic.go:356] "Generic (PLEG): container finished" podID="35fcc59c-95e0-4dce-9598-0bae15408208" containerID="6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11" exitCode=0 Jul 08 21:51:25.608679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.608489 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" Jul 08 21:51:25.608679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.608538 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" event={"ID":"35fcc59c-95e0-4dce-9598-0bae15408208","Type":"ContainerDied","Data":"6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11"} Jul 08 21:51:25.608679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.608580 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2" event={"ID":"35fcc59c-95e0-4dce-9598-0bae15408208","Type":"ContainerDied","Data":"39026a0caccb5b252abce7e3f2e2ddd0932b520f48bd2ebd895f82a5a7a8f926"} Jul 08 21:51:25.608679 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.608602 2578 scope.go:117] "RemoveContainer" containerID="6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11" Jul 08 21:51:25.618554 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.618535 2578 scope.go:117] "RemoveContainer" containerID="6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11" Jul 08 21:51:25.618796 ip-10-0-138-169 kubenswrapper[2578]: E0708 21:51:25.618777 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11\": container with ID starting with 6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11 not found: ID does not exist" containerID="6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11" Jul 08 21:51:25.618835 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.618804 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11"} err="failed to get container status \"6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11\": rpc error: code = NotFound desc = could not find container \"6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11\": container with ID starting with 6ff0d040159923e033e269fa9f44598cfe614508438053bef5131cd017bffc11 not found: ID does not exist" Jul 08 21:51:25.630699 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.630676 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2"] Jul 08 21:51:25.632725 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:25.632706 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9b1f52/test-model-9b1f52-kserve-65766bc5ff-w9fq2"] Jul 08 21:51:26.616225 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:26.616195 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="161a0b79-71c8-47c7-95ed-122d9bb1fbd8" path="/var/lib/kubelet/pods/161a0b79-71c8-47c7-95ed-122d9bb1fbd8/volumes" Jul 08 21:51:26.616602 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:26.616581 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35fcc59c-95e0-4dce-9598-0bae15408208" path="/var/lib/kubelet/pods/35fcc59c-95e0-4dce-9598-0bae15408208/volumes" Jul 08 21:51:56.631542 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:51:56.631513 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:52:12.222996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:52:12.222959 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:52:28.623555 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:52:28.623524 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:52:43.024503 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:52:43.024470 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:52:53.027445 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:52:53.027411 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:54:42.812120 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:54:42.812052 2578 scope.go:117] "RemoveContainer" containerID="308e4f499d467e96992366f2c5483d4664eee55eb0176e3b3b18638c69a5ed32" Jul 08 21:55:42.775918 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:55:42.775889 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:55:42.793304 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:55:42.793276 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:56:03.432902 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:03.432800 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:56:13.327207 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:13.327176 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54f54b8558-jqkxk_3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13/authorino/0.log" Jul 08 21:56:15.331811 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:15.331777 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wcqv4"] Jul 08 21:56:18.714860 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:18.714835 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-wpvbh_b15932d2-1e45-43d2-a3eb-c0bcfb85746d/copy-manifests/0.log" Jul 08 21:56:18.721996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:18.721974 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-wpvbh_b15932d2-1e45-43d2-a3eb-c0bcfb85746d/manager/0.log" Jul 08 21:56:19.650778 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.650754 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c_aba21b3d-8986-4971-9ae8-9d877496e7ff/pull/0.log" Jul 08 21:56:19.656056 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.656035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c_aba21b3d-8986-4971-9ae8-9d877496e7ff/extract/0.log" Jul 08 21:56:19.661670 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.661653 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c_aba21b3d-8986-4971-9ae8-9d877496e7ff/util/0.log" Jul 08 21:56:19.780805 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.780777 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs_eaabac55-ab9d-46d3-885c-5f50e283c6df/pull/0.log" Jul 08 21:56:19.787104 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.787087 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs_eaabac55-ab9d-46d3-885c-5f50e283c6df/extract/0.log" Jul 08 21:56:19.792703 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.792684 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs_eaabac55-ab9d-46d3-885c-5f50e283c6df/util/0.log" Jul 08 21:56:19.906574 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.906505 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69_cbd54043-12f9-4647-8b97-48586f7badbd/util/0.log" Jul 08 21:56:19.912824 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.912802 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69_cbd54043-12f9-4647-8b97-48586f7badbd/pull/0.log" Jul 08 21:56:19.918480 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:19.918450 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69_cbd54043-12f9-4647-8b97-48586f7badbd/extract/0.log" Jul 08 21:56:20.035774 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:20.035752 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7_96867611-09a8-44bf-91ea-87c2603d6cdc/extract/0.log" Jul 08 21:56:20.041769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:20.041744 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7_96867611-09a8-44bf-91ea-87c2603d6cdc/util/0.log" Jul 08 21:56:20.047201 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:20.047171 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7_96867611-09a8-44bf-91ea-87c2603d6cdc/pull/0.log" Jul 08 21:56:20.161082 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:20.160988 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54f54b8558-jqkxk_3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13/authorino/0.log" Jul 08 21:56:20.522642 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:20.522565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2z6gd_dc342ffe-374f-4ae8-b458-3f911cb55fa4/kuadrant-console-plugin/0.log" Jul 08 21:56:20.883211 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:20.883183 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wcqv4_ced0dc15-2026-4f86-9278-40b95abfde23/limitador/0.log" Jul 08 21:56:21.618799 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:21.618774 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-s8p92_aa85c81a-9b7c-4661-b0cb-9ab227f37d7f/kube-auth-proxy/0.log" Jul 08 21:56:23.139076 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:23.139043 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vzdmm_579aea95-e632-4a1a-83da-dfc93e1cbe82/main/0.log" Jul 08 21:56:27.376815 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.376787 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ccpfm/must-gather-hp2gc"] Jul 08 21:56:27.377208 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.377167 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fcc59c-95e0-4dce-9598-0bae15408208" containerName="main" Jul 08 21:56:27.377208 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.377178 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fcc59c-95e0-4dce-9598-0bae15408208" containerName="main" Jul 08 21:56:27.377208 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.377197 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="161a0b79-71c8-47c7-95ed-122d9bb1fbd8" containerName="main" Jul 08 21:56:27.377208 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.377202 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="161a0b79-71c8-47c7-95ed-122d9bb1fbd8" containerName="main" Jul 08 21:56:27.377341 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.377269 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="35fcc59c-95e0-4dce-9598-0bae15408208" containerName="main" Jul 08 21:56:27.377341 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.377277 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="161a0b79-71c8-47c7-95ed-122d9bb1fbd8" containerName="main" Jul 08 21:56:27.380475 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.380458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.383571 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.383549 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ccpfm\"/\"openshift-service-ca.crt\"" Jul 08 21:56:27.383571 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.383560 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ccpfm\"/\"kube-root-ca.crt\"" Jul 08 21:56:27.392205 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.392185 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ccpfm/must-gather-hp2gc"] Jul 08 21:56:27.408036 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.408009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/148fc8fd-591a-41a2-a5f6-2f16e2bf8307-must-gather-output\") pod \"must-gather-hp2gc\" (UID: \"148fc8fd-591a-41a2-a5f6-2f16e2bf8307\") " pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.408158 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.408059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jt75\" (UniqueName: \"kubernetes.io/projected/148fc8fd-591a-41a2-a5f6-2f16e2bf8307-kube-api-access-6jt75\") pod \"must-gather-hp2gc\" (UID: \"148fc8fd-591a-41a2-a5f6-2f16e2bf8307\") " pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.509020 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.508988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/148fc8fd-591a-41a2-a5f6-2f16e2bf8307-must-gather-output\") pod \"must-gather-hp2gc\" (UID: \"148fc8fd-591a-41a2-a5f6-2f16e2bf8307\") " pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.509146 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.509035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jt75\" (UniqueName: \"kubernetes.io/projected/148fc8fd-591a-41a2-a5f6-2f16e2bf8307-kube-api-access-6jt75\") pod \"must-gather-hp2gc\" (UID: \"148fc8fd-591a-41a2-a5f6-2f16e2bf8307\") " pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.509485 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.509461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/148fc8fd-591a-41a2-a5f6-2f16e2bf8307-must-gather-output\") pod \"must-gather-hp2gc\" (UID: \"148fc8fd-591a-41a2-a5f6-2f16e2bf8307\") " pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.517634 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.517607 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jt75\" (UniqueName: \"kubernetes.io/projected/148fc8fd-591a-41a2-a5f6-2f16e2bf8307-kube-api-access-6jt75\") pod \"must-gather-hp2gc\" (UID: \"148fc8fd-591a-41a2-a5f6-2f16e2bf8307\") " pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.690077 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.689999 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ccpfm/must-gather-hp2gc" Jul 08 21:56:27.821346 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.821320 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ccpfm/must-gather-hp2gc"] Jul 08 21:56:27.822832 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:56:27.822803 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod148fc8fd_591a_41a2_a5f6_2f16e2bf8307.slice/crio-7c5b2844be5cabf522fc4f7b9b07e09d96fd7dd4fb4a4f369168dded74e58607 WatchSource:0}: Error finding container 7c5b2844be5cabf522fc4f7b9b07e09d96fd7dd4fb4a4f369168dded74e58607: Status 404 returned error can't find the container with id 7c5b2844be5cabf522fc4f7b9b07e09d96fd7dd4fb4a4f369168dded74e58607 Jul 08 21:56:27.824559 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:27.824534 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:56:28.274370 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:28.274305 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/must-gather-hp2gc" event={"ID":"148fc8fd-591a-41a2-a5f6-2f16e2bf8307","Type":"ContainerStarted","Data":"7c5b2844be5cabf522fc4f7b9b07e09d96fd7dd4fb4a4f369168dded74e58607"} Jul 08 21:56:29.284223 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:29.284180 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/must-gather-hp2gc" event={"ID":"148fc8fd-591a-41a2-a5f6-2f16e2bf8307","Type":"ContainerStarted","Data":"78aa3b0d5a52045d317a99b6bfb27001527040fd6050c45233e64114a42b8442"} Jul 08 21:56:29.284699 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:29.284234 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/must-gather-hp2gc" event={"ID":"148fc8fd-591a-41a2-a5f6-2f16e2bf8307","Type":"ContainerStarted","Data":"e9dd3478cddd7c6961fbcd0a9bdb7250ca4877bd9eed87723ddaf22f1b86f120"} Jul 08 21:56:29.301055 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:29.300999 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ccpfm/must-gather-hp2gc" podStartSLOduration=1.416772302 podStartE2EDuration="2.300978037s" totalImagesPullingTime="884.205735ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:56:29.30021418 +0000 UTC m=+2447.210651598" watchObservedRunningTime="2026-07-08 21:56:29.300978037 +0000 UTC m=+2447.211415455" Jul 08 21:56:30.343254 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:30.343217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zjwzd_fb36b861-2ad4-4f39-8bf3-59e5f9a76e9e/global-pull-secret-syncer/0.log" Jul 08 21:56:30.426717 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:30.426688 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fvs79_63a1a579-bb03-4938-bec5-a6c6231fb30b/konnectivity-agent/0.log" Jul 08 21:56:30.509690 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:30.509663 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-169.ec2.internal_cacf4868144cd01ddbdab4f36e2a7a87/haproxy/0.log" Jul 08 21:56:35.165109 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.165043 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c_aba21b3d-8986-4971-9ae8-9d877496e7ff/extract/0.log" Jul 08 21:56:35.193236 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.193212 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c_aba21b3d-8986-4971-9ae8-9d877496e7ff/util/0.log" Jul 08 21:56:35.220084 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.220055 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599pr5c_aba21b3d-8986-4971-9ae8-9d877496e7ff/pull/0.log" Jul 08 21:56:35.247603 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.247576 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs_eaabac55-ab9d-46d3-885c-5f50e283c6df/extract/0.log" Jul 08 21:56:35.270175 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.270145 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs_eaabac55-ab9d-46d3-885c-5f50e283c6df/util/0.log" Jul 08 21:56:35.292146 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.292114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0wtmhs_eaabac55-ab9d-46d3-885c-5f50e283c6df/pull/0.log" Jul 08 21:56:35.321900 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.321833 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69_cbd54043-12f9-4647-8b97-48586f7badbd/extract/0.log" Jul 08 21:56:35.343935 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.343912 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69_cbd54043-12f9-4647-8b97-48586f7badbd/util/0.log" Jul 08 21:56:35.363844 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.363797 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qkr69_cbd54043-12f9-4647-8b97-48586f7badbd/pull/0.log" Jul 08 21:56:35.392933 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.392906 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7_96867611-09a8-44bf-91ea-87c2603d6cdc/extract/0.log" Jul 08 21:56:35.416586 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.416513 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7_96867611-09a8-44bf-91ea-87c2603d6cdc/util/0.log" Jul 08 21:56:35.442976 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.442924 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z4bn7_96867611-09a8-44bf-91ea-87c2603d6cdc/pull/0.log" Jul 08 21:56:35.819939 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.818973 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54f54b8558-jqkxk_3ac24fa7-ea8a-4c99-95a2-1fab5e1e3c13/authorino/0.log" Jul 08 21:56:35.906385 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:35.906344 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2z6gd_dc342ffe-374f-4ae8-b458-3f911cb55fa4/kuadrant-console-plugin/0.log" Jul 08 21:56:36.086226 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:36.086155 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wcqv4_ced0dc15-2026-4f86-9278-40b95abfde23/limitador/0.log" Jul 08 21:56:37.753842 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:37.753810 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/alertmanager/0.log" Jul 08 21:56:37.790395 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:37.790347 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/config-reloader/0.log" Jul 08 21:56:37.832584 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:37.832557 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/kube-rbac-proxy-web/0.log" Jul 08 21:56:37.885831 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:37.885806 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/kube-rbac-proxy/0.log" Jul 08 21:56:37.933147 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:37.933068 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/kube-rbac-proxy-metric/0.log" Jul 08 21:56:37.966022 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:37.965946 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/prom-label-proxy/0.log" Jul 08 21:56:38.002518 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.002465 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_086c9898-c0e9-4ef1-8379-61b44ae0a57d/init-config-reloader/0.log" Jul 08 21:56:38.095996 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.095964 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-n9b5x_3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48/kube-state-metrics/0.log" Jul 08 21:56:38.132553 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.132513 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-n9b5x_3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48/kube-rbac-proxy-main/0.log" Jul 08 21:56:38.169599 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.169558 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-n9b5x_3a2ef51a-46e3-4f6b-a9f1-ce1b165faa48/kube-rbac-proxy-self/0.log" Jul 08 21:56:38.251420 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.251281 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-lbl2s_77314b41-10fb-4a07-b2a0-f0f466e1b40d/monitoring-plugin/0.log" Jul 08 21:56:38.384576 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.384548 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mw5xx_b8c113b8-6008-47e9-a6a7-1845b08cf79c/node-exporter/0.log" Jul 08 21:56:38.420992 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.420966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mw5xx_b8c113b8-6008-47e9-a6a7-1845b08cf79c/kube-rbac-proxy/0.log" Jul 08 21:56:38.456179 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.456144 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mw5xx_b8c113b8-6008-47e9-a6a7-1845b08cf79c/init-textfile/0.log" Jul 08 21:56:38.839271 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.839193 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-bq65z_2bc8804c-a59d-4446-9e53-b4fbbded9d82/prometheus-operator/0.log" Jul 08 21:56:38.857252 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.857225 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-bq65z_2bc8804c-a59d-4446-9e53-b4fbbded9d82/kube-rbac-proxy/0.log" Jul 08 21:56:38.883436 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.883401 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-n92dd_ed5b64d4-2970-4e73-8254-9b2cd83969e7/prometheus-operator-admission-webhook/0.log" Jul 08 21:56:38.910918 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.910882 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-67cfbd585c-6277f_0e605723-b6fb-4323-a009-2ad2e3bbb739/telemeter-client/0.log" Jul 08 21:56:38.934417 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.934389 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-67cfbd585c-6277f_0e605723-b6fb-4323-a009-2ad2e3bbb739/reload/0.log" Jul 08 21:56:38.956240 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.956213 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-67cfbd585c-6277f_0e605723-b6fb-4323-a009-2ad2e3bbb739/kube-rbac-proxy/0.log" Jul 08 21:56:38.988747 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:38.988718 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-98fbc787f-2kh9s_ea6c37a5-4460-4ab5-9bfb-709395e7ee06/thanos-query/0.log" Jul 08 21:56:39.009652 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:39.009617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-98fbc787f-2kh9s_ea6c37a5-4460-4ab5-9bfb-709395e7ee06/kube-rbac-proxy-web/0.log" Jul 08 21:56:39.031211 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:39.031175 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-98fbc787f-2kh9s_ea6c37a5-4460-4ab5-9bfb-709395e7ee06/kube-rbac-proxy/0.log" Jul 08 21:56:39.052006 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:39.051978 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-98fbc787f-2kh9s_ea6c37a5-4460-4ab5-9bfb-709395e7ee06/prom-label-proxy/0.log" Jul 08 21:56:39.075308 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:39.075262 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-98fbc787f-2kh9s_ea6c37a5-4460-4ab5-9bfb-709395e7ee06/kube-rbac-proxy-rules/0.log" Jul 08 21:56:39.096425 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:39.096347 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-98fbc787f-2kh9s_ea6c37a5-4460-4ab5-9bfb-709395e7ee06/kube-rbac-proxy-metrics/0.log" Jul 08 21:56:40.217310 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.217280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-tk5vf_b392e7e5-7359-4906-bec1-134f21238c3a/networking-console-plugin/0.log" Jul 08 21:56:40.328573 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.328538 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g"] Jul 08 21:56:40.337648 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.337620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.342703 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.342672 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g"] Jul 08 21:56:40.344345 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.344329 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ccpfm\"/\"default-dockercfg-v6cws\"" Jul 08 21:56:40.448321 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.448263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gczwx\" (UniqueName: \"kubernetes.io/projected/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-kube-api-access-gczwx\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.448321 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.448324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-sys\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.448577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.448349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-proc\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.448577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.448459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-lib-modules\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.448577 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.448515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-podres\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.549991 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.549901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-lib-modules\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550418 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550317 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-lib-modules\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550523 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-podres\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550523 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550468 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-podres\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550619 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gczwx\" (UniqueName: \"kubernetes.io/projected/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-kube-api-access-gczwx\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550619 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-sys\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550619 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-proc\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-sys\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.550769 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.550724 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-proc\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.560060 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.558915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gczwx\" (UniqueName: \"kubernetes.io/projected/7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4-kube-api-access-gczwx\") pod \"perf-node-gather-daemonset-rvq5g\" (UID: \"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.655632 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.655597 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:40.838723 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.838644 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/2.log" Jul 08 21:56:40.840201 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.840176 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g"] Jul 08 21:56:40.842909 ip-10-0-138-169 kubenswrapper[2578]: W0708 21:56:40.842826 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7fa0b434_b6eb_43ca_8fb0_3aa2157b7ee4.slice/crio-1f4b66db6a8d5dace9dc103142eb6544a77d61afdb8136335ab663454b5c5515 WatchSource:0}: Error finding container 1f4b66db6a8d5dace9dc103142eb6544a77d61afdb8136335ab663454b5c5515: Status 404 returned error can't find the container with id 1f4b66db6a8d5dace9dc103142eb6544a77d61afdb8136335ab663454b5c5515 Jul 08 21:56:40.846417 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:40.846180 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-v852d_32c2588b-239f-48bb-8565-cadad7a589a2/console-operator/3.log" Jul 08 21:56:41.338383 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:41.338351 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-wc7hd_c04e0a32-8bd7-46cd-9435-0f74a7adb781/download-server/0.log" Jul 08 21:56:41.385069 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:41.385029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" event={"ID":"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4","Type":"ContainerStarted","Data":"2b9ee471f5073e7ec8b204923f10269e62b39eff2e04d0e124a6b99abf92f51a"} Jul 08 21:56:41.385256 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:41.385096 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" event={"ID":"7fa0b434-b6eb-43ca-8fb0-3aa2157b7ee4","Type":"ContainerStarted","Data":"1f4b66db6a8d5dace9dc103142eb6544a77d61afdb8136335ab663454b5c5515"} Jul 08 21:56:41.385961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:41.385936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:42.984553 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:42.984526 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gfqn2_5d640c3d-da59-4e06-880a-e1b27b2f6013/dns/0.log" Jul 08 21:56:43.007601 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:43.007570 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gfqn2_5d640c3d-da59-4e06-880a-e1b27b2f6013/kube-rbac-proxy/0.log" Jul 08 21:56:43.135936 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:43.135906 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wlhfz_3c83b512-6ff0-402d-85a9-44883cb68f6a/dns-node-resolver/0.log" Jul 08 21:56:43.738665 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:43.738641 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zks8d_52b53d04-7d5b-4a95-ba7a-ea8ad96f1e63/node-ca/0.log" Jul 08 21:56:44.889418 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:44.889390 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-s8p92_aa85c81a-9b7c-4661-b0cb-9ab227f37d7f/kube-auth-proxy/0.log" Jul 08 21:56:45.805503 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:45.805474 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wznnp_0235050f-9f13-48dd-93a1-d180b0fd9a21/serve-healthcheck-canary/0.log" Jul 08 21:56:46.287403 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:46.287374 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-f85dw_006b5ab4-e339-4591-ba91-124fcf05b773/insights-operator/0.log" Jul 08 21:56:46.288176 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:46.288158 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-f85dw_006b5ab4-e339-4591-ba91-124fcf05b773/insights-operator/1.log" Jul 08 21:56:46.381842 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:46.381819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9thh_a2d4bdc7-b04d-4556-96af-87e98ad7d4de/kube-rbac-proxy/0.log" Jul 08 21:56:46.402789 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:46.402759 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9thh_a2d4bdc7-b04d-4556-96af-87e98ad7d4de/exporter/0.log" Jul 08 21:56:46.432926 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:46.432894 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9thh_a2d4bdc7-b04d-4556-96af-87e98ad7d4de/extractor/0.log" Jul 08 21:56:48.412745 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:48.412719 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" Jul 08 21:56:48.434072 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:48.434010 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-rvq5g" podStartSLOduration=8.433989589 podStartE2EDuration="8.433989589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:56:41.404068721 +0000 UTC m=+2459.314506138" watchObservedRunningTime="2026-07-08 21:56:48.433989589 +0000 UTC m=+2466.344427007" Jul 08 21:56:49.124677 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:49.124644 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-wpvbh_b15932d2-1e45-43d2-a3eb-c0bcfb85746d/manager/0.log" Jul 08 21:56:49.144763 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:49.144734 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-wpvbh_b15932d2-1e45-43d2-a3eb-c0bcfb85746d/copy-manifests/0.log" Jul 08 21:56:50.535723 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:50.535698 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5cdf98c45c-5b5hq_a18c2353-0bc5-49d6-a36d-cdbca143400a/manager/0.log" Jul 08 21:56:50.635961 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:50.635926 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-s8p92_aa85c81a-9b7c-4661-b0cb-9ab227f37d7f/kube-auth-proxy/0.log" Jul 08 21:56:52.259400 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:52.259358 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-s8p92_aa85c81a-9b7c-4661-b0cb-9ab227f37d7f/kube-auth-proxy/0.log" Jul 08 21:56:55.648430 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:55.648397 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-h7jkn_f3c61254-5d58-41d7-bb44-ab2f04cf502a/migrator/0.log" Jul 08 21:56:55.671563 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:55.671541 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-h7jkn_f3c61254-5d58-41d7-bb44-ab2f04cf502a/graceful-termination/0.log" Jul 08 21:56:56.021139 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:56.021065 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-t22lh_6df65827-dae8-46cf-a3c3-dab3652ffe06/kube-storage-version-migrator-operator/1.log" Jul 08 21:56:56.022053 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:56.022026 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-t22lh_6df65827-dae8-46cf-a3c3-dab3652ffe06/kube-storage-version-migrator-operator/0.log" Jul 08 21:56:57.138366 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.138332 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-82fgn_40093444-e186-40d8-95d8-f48c95daceb7/kube-multus/0.log" Jul 08 21:56:57.165088 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.165064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/kube-multus-additional-cni-plugins/0.log" Jul 08 21:56:57.188446 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.188398 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/egress-router-binary-copy/0.log" Jul 08 21:56:57.212353 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.212322 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/cni-plugins/0.log" Jul 08 21:56:57.234171 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.234149 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/bond-cni-plugin/0.log" Jul 08 21:56:57.259150 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.259133 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/routeoverride-cni/0.log" Jul 08 21:56:57.282589 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.282571 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/whereabouts-cni-bincopy/0.log" Jul 08 21:56:57.304812 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.304783 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nfrtf_66556e1d-1cf1-4073-bb17-f22eb8835d92/whereabouts-cni/0.log" Jul 08 21:56:57.741405 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.741380 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gpq8z_0ebaf425-2624-4be4-b8e3-0676363ddf45/network-metrics-daemon/0.log" Jul 08 21:56:57.761184 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:57.761159 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gpq8z_0ebaf425-2624-4be4-b8e3-0676363ddf45/kube-rbac-proxy/0.log" Jul 08 21:56:58.840233 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:58.840205 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pt5h2_b4fa8fb4-20f2-4200-bf3c-4eabb8024214/ovn-controller/0.log" Jul 08 21:56:58.859518 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:58.859490 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pt5h2_b4fa8fb4-20f2-4200-bf3c-4eabb8024214/ovn-acl-logging/0.log" Jul 08 21:56:58.889497 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:58.889469 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pt5h2_b4fa8fb4-20f2-4200-bf3c-4eabb8024214/kube-rbac-proxy-node/0.log" Jul 08 21:56:58.924398 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:58.924374 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pt5h2_b4fa8fb4-20f2-4200-bf3c-4eabb8024214/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 21:56:58.946915 ip-10-0-138-169 kubenswrapper[2578]: I0708 21:56:58.946890 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pt5h2_b4fa8fb4-20f2-4200-bf3c-4eabb8024214/northd/0.log"