Jul 08 09:07:00.437286 ip-10-0-140-252 systemd[1]: Starting Kubernetes Kubelet... Jul 08 09:07:01.909758 ip-10-0-140-252 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 09:07:01.909758 ip-10-0-140-252 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 09:07:01.909758 ip-10-0-140-252 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 09:07:01.909758 ip-10-0-140-252 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 09:07:01.909758 ip-10-0-140-252 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 09:07:01.910739 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.910653 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 09:07:01.914162 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914146 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:07:01.914162 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914162 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:07:01.914162 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914166 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914169 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914172 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914175 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914178 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914181 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914184 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914186 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914189 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914192 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914194 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914197 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914199 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914202 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914205 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914208 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914210 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914213 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914225 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:07:01.914261 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914231 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914235 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914238 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914241 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914244 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914247 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914250 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914253 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914255 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914258 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914260 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914263 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914266 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914269 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914271 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914275 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914277 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914280 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914282 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914285 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:07:01.914709 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914287 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914290 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914293 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914296 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914298 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914301 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914303 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914305 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914308 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914310 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914313 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914315 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914323 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914326 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914332 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914335 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914339 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914341 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914344 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:07:01.915315 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914347 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914350 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914352 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914355 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914358 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914360 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914363 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914365 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914368 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914370 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914373 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914375 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914378 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914380 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914383 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914385 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914388 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914390 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914393 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914395 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:07:01.915768 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914398 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914400 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914403 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914405 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914408 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914410 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914419 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914424 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914427 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914430 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914432 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914435 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914437 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914440 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914442 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914445 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914447 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914449 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:07:01.916262 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.914452 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916666 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916673 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916677 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916680 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916683 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916686 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916689 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916691 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916695 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916698 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916700 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:07:01.916698 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916703 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916706 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916709 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916712 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916714 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916717 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916719 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916722 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916724 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916728 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916730 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916733 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916735 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916738 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916740 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916743 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916745 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916748 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916750 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916752 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:07:01.917007 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916755 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916757 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916760 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916762 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916765 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916767 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916770 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916772 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916775 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916778 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916780 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916783 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916785 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916787 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916790 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916793 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916796 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916798 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916801 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916803 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:07:01.917514 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916806 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916808 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916811 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916814 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916817 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916821 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916824 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916827 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916830 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916833 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916835 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916838 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916840 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916844 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916848 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916851 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916854 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916856 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916860 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:07:01.918009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916862 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916865 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916869 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916871 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916874 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916876 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916879 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916882 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916885 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916888 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916892 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916895 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916913 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916916 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916919 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916921 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916924 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916927 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916930 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:07:01.918495 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916932 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916935 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916938 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916940 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916943 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916945 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916948 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916950 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916953 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.916955 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917327 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917332 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917335 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917338 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917341 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917348 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917351 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917353 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917356 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917358 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:07:01.918974 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917361 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917364 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917367 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917370 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917373 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917376 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917379 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917381 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917384 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917387 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917390 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917393 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917395 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917397 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917400 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917402 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917405 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917407 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917410 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917412 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:07:01.919463 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917415 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917418 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917420 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917423 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917426 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917428 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917431 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917433 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917436 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917439 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917442 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917444 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917447 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917449 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917452 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917454 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917457 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917460 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917462 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917465 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917467 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:07:01.920003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917470 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917473 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917477 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917479 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917481 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917484 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917486 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917489 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917491 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917494 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917498 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917502 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917508 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917511 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917513 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917516 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917519 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917521 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917524 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917526 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:07:01.920500 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917529 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917531 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917534 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917537 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917539 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917542 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917544 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917549 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917552 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917555 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917558 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917561 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917564 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917567 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917574 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917576 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917579 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917582 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:07:01.921012 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917584 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917587 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917590 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917593 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917595 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917598 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917601 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917604 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917606 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.917609 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917690 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917696 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917703 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917708 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917712 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917715 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917719 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917723 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917727 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917730 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917733 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 09:07:01.921490 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917736 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917739 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917742 2572 flags.go:64] FLAG: --cgroup-root="" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917746 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917749 2572 flags.go:64] FLAG: --client-ca-file="" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917751 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.917754 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918175 2572 flags.go:64] FLAG: --cluster-domain="" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918180 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918184 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918187 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918191 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918195 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918199 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918202 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918206 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918209 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918211 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918214 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918217 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918220 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918224 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918227 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918230 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918233 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 09:07:01.922001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918236 2572 flags.go:64] FLAG: --enable-server="true" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918239 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918244 2572 flags.go:64] FLAG: --event-burst="100" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918247 2572 flags.go:64] FLAG: --event-qps="50" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918250 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918253 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918256 2572 flags.go:64] FLAG: --eviction-hard="" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918259 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918262 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918265 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918268 2572 flags.go:64] FLAG: --eviction-soft="" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918271 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918274 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918277 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918280 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918283 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918286 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918289 2572 flags.go:64] FLAG: --feature-gates="" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918293 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918296 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918300 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918303 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918306 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918309 2572 flags.go:64] FLAG: --help="false" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918312 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-140-252.ec2.internal" Jul 08 09:07:01.922594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918314 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918318 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918321 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918324 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918327 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918330 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918333 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918336 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918339 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918342 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918345 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918348 2572 flags.go:64] FLAG: --kube-reserved="" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918350 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918353 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918356 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918359 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918362 2572 flags.go:64] FLAG: --lock-file="" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918365 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918367 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918370 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918375 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918378 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918382 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 09:07:01.923220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918384 2572 flags.go:64] FLAG: --logging-format="text" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918388 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918391 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918394 2572 flags.go:64] FLAG: --manifest-url="" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918398 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918403 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918406 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918410 2572 flags.go:64] FLAG: --max-pods="110" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918413 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918416 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918419 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918422 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918425 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918428 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918431 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918438 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918441 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918444 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918447 2572 flags.go:64] FLAG: --pod-cidr="" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918450 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918455 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918458 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918461 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918464 2572 flags.go:64] FLAG: --port="10250" Jul 08 09:07:01.923786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918467 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918470 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-067dc82fd9484b177" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918473 2572 flags.go:64] FLAG: --qos-reserved="" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918476 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918479 2572 flags.go:64] FLAG: --register-node="true" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918482 2572 flags.go:64] FLAG: --register-with-taints="" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918485 2572 flags.go:64] FLAG: --registry-burst="10" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918488 2572 flags.go:64] FLAG: --registry-qps="5" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918492 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918495 2572 flags.go:64] FLAG: --reserved-memory="" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918499 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918503 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918506 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918509 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918512 2572 flags.go:64] FLAG: --runonce="false" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918515 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918518 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918522 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918524 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918527 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918530 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918533 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918536 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918539 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918542 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 09:07:01.924449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918545 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918548 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918551 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918554 2572 flags.go:64] FLAG: --system-cgroups="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918557 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918562 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918564 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918567 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918571 2572 flags.go:64] FLAG: --tls-min-version="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918573 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918576 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918579 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918582 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918584 2572 flags.go:64] FLAG: --v="2" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918588 2572 flags.go:64] FLAG: --version="false" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918593 2572 flags.go:64] FLAG: --vmodule="" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918597 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.918600 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918714 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918718 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918721 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918724 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918727 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918730 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:07:01.925110 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918732 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918735 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918737 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918740 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918742 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918745 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918748 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918750 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918753 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918755 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918758 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918760 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918763 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918766 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918768 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918771 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918773 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918777 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918780 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918783 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:07:01.926096 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918786 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918789 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918793 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918798 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918801 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918804 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918807 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918810 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918814 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918816 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918820 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918822 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918825 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918828 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918830 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918833 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918835 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918838 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918840 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:07:01.926751 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918843 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918845 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918848 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918850 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918853 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918855 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918858 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918860 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918863 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918865 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918868 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918870 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918873 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918875 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918878 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918880 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918884 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918887 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918889 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918892 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:07:01.927230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918895 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918911 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918914 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918917 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918919 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918922 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918924 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918927 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918930 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918932 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918935 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918937 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918939 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918942 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918945 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918947 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918950 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918952 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918955 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918957 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:07:01.927722 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918959 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918962 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918964 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918967 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918969 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918972 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918974 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918977 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918981 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918984 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918986 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918992 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918994 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.918997 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.919516 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 09:07:01.928242 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.927610 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.927626 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.928286 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.928294 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928356 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928361 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928365 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928368 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928371 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928374 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928377 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928379 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928382 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928384 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928387 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928389 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928392 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928395 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928397 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:07:01.928626 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928400 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928403 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928405 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928408 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928411 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928413 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928416 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928419 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928421 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928424 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928426 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928430 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928434 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928437 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928440 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928444 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928447 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928450 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928453 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:07:01.929114 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928456 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928459 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928461 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928465 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928468 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928471 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928474 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928477 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928480 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928483 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928485 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928488 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928491 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928494 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928497 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928500 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928503 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928505 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928509 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928511 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:07:01.929579 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928514 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928517 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928519 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928522 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928524 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928526 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928529 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928532 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928534 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928537 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928539 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928542 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928545 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928547 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928550 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928552 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928555 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928558 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928560 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928562 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:07:01.930074 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928565 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928569 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928572 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928574 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928577 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928580 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928583 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928585 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928588 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928590 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928593 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928595 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928598 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928600 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928603 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928606 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928608 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928611 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928613 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928616 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:07:01.930559 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928629 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928633 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928636 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928638 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928641 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.928647 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928776 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928781 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928784 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928786 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928789 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928792 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928795 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928797 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928800 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 09:07:01.931086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928802 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928805 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928808 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928812 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928815 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928817 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928820 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928822 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928825 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928827 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928831 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928835 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928838 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928841 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928844 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928847 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928850 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928852 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928855 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 09:07:01.931445 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928857 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928859 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928862 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928865 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928867 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928870 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928872 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928875 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928877 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928880 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928882 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928885 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928887 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928890 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928892 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928895 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928910 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928916 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928919 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928922 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 09:07:01.931911 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928925 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928927 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928930 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928932 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928935 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928937 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928940 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928942 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928945 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928947 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928950 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928952 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928955 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928957 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928960 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928962 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928965 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928967 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928969 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928972 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 09:07:01.932397 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928974 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928977 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928979 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928982 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928984 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928987 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928989 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928991 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928994 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.928998 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929002 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929005 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929007 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929010 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929013 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929015 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929018 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929020 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929022 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 09:07:01.932918 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929025 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929027 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929030 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929032 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929035 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929037 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929040 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929043 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929047 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929050 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929052 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:01.929055 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.929060 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.929182 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 09:07:01.933387 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.931234 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 09:07:01.933732 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.932547 2572 server.go:1019] "Starting client certificate rotation" Jul 08 09:07:01.933732 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.932652 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 09:07:01.933732 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.932683 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 09:07:01.935253 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.935239 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 09:07:01.940795 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.940777 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 09:07:01.951104 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.951086 2572 log.go:25] "Validated CRI v1 runtime API" Jul 08 09:07:01.954344 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.954331 2572 log.go:25] "Validated CRI v1 image API" Jul 08 09:07:01.955268 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.955249 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 09:07:01.958160 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.958135 2572 fs.go:135] Filesystem UUIDs: map[3b8d633b-1aa5-4aac-b373-bc3e1d2aaaec:/dev/nvme0n1p3 576161a6-516d-4c8f-8c84-26079c162718:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 08 09:07:01.958205 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.958161 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 09:07:01.961497 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.961482 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 09:07:01.963657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.963540 2572 manager.go:216] Machine: {Timestamp:2026-07-08 09:07:01.962365768 +0000 UTC m=+0.317377320 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101610 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec290395cb9000ce1c8799e41bbeec09 SystemUUID:ec290395-cb90-00ce-1c87-99e41bbeec09 BootID:309c54f0-8430-4310-ad65-86d331ea71df Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c6:1e:7c:0d:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c6:1e:7c:0d:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:03:72:65:a5:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 09:07:01.963657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.963654 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 09:07:01.963753 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.963735 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 09:07:01.966465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.966435 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 09:07:01.966608 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.966468 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-252.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 09:07:01.967146 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.967136 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 09:07:01.967184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.967149 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 09:07:01.967184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.967157 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 09:07:01.967718 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.967709 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 09:07:01.968909 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.968887 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 09:07:01.971524 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.971512 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 08 09:07:01.971648 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.971639 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 09:07:01.972162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972151 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 08 09:07:01.972202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972171 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 09:07:01.972202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972195 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 09:07:01.972261 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972203 2572 kubelet.go:398] "Adding apiserver pod source" Jul 08 09:07:01.972261 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972212 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 09:07:01.972683 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972659 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 09:07:01.972683 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.972685 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 09:07:01.975994 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.975970 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 09:07:01.977614 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.977596 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 09:07:01.977693 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.977628 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 09:07:01.978668 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978656 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978673 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978680 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978686 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978695 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978700 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978708 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 09:07:01.978714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978713 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 09:07:01.978945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978720 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 09:07:01.978945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978725 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 09:07:01.978945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978733 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 09:07:01.978945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.978738 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 09:07:01.979356 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.979346 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 09:07:01.979356 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.979356 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 09:07:01.982618 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.982605 2572 server.go:1268] "Started kubelet" Jul 08 09:07:01.982756 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.982699 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 09:07:01.982859 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.982772 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 09:07:01.982859 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.982797 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 09:07:01.983328 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.983308 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-252.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 09:07:01.983587 ip-10-0-140-252 systemd[1]: Started Kubernetes Kubelet. Jul 08 09:07:01.983707 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.983651 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-252.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 09:07:01.983863 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.983834 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 09:07:01.983978 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.983956 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 09:07:01.985088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.985071 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 08 09:07:01.987812 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.986977 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-252.ec2.internal.18c045469a9a5e78 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-252.ec2.internal,UID:ip-10-0-140-252.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-252.ec2.internal,},FirstTimestamp:2026-07-08 09:07:01.982551672 +0000 UTC m=+0.337563225,LastTimestamp:2026-07-08 09:07:01.982551672 +0000 UTC m=+0.337563225,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-252.ec2.internal,}" Jul 08 09:07:01.989442 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.989419 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 09:07:01.989982 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.989963 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 09:07:01.989982 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.989977 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 09:07:01.990136 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.990033 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:01.990550 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990530 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 09:07:01.990550 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990549 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 09:07:01.990666 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990647 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 08 09:07:01.990666 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990652 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 08 09:07:01.990753 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990713 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 09:07:01.990856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990836 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 09:07:01.990856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990856 2572 factory.go:55] Registering systemd factory Jul 08 09:07:01.991006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.990865 2572 factory.go:223] Registration of the systemd container factory successfully Jul 08 09:07:01.991149 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.991132 2572 factory.go:153] Registering CRI-O factory Jul 08 09:07:01.991196 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.991155 2572 factory.go:223] Registration of the crio container factory successfully Jul 08 09:07:01.991196 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.991180 2572 factory.go:103] Registering Raw factory Jul 08 09:07:01.991266 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.991204 2572 manager.go:1196] Started watching for new ooms in manager Jul 08 09:07:01.991867 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.991852 2572 manager.go:349] Starting recovery of all containers Jul 08 09:07:01.992229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.992197 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hv7kz" Jul 08 09:07:01.992610 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.992579 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-252.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 09:07:01.992718 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:01.992694 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 09:07:01.999293 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:01.999266 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hv7kz" Jul 08 09:07:02.002996 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.002765 2572 manager.go:354] Recovery completed Jul 08 09:07:02.007817 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.007804 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.009593 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.009578 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.009650 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.009607 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.009650 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.009618 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.010240 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010217 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 09:07:02.010240 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010234 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.010240 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010239 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 09:07:02.010401 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010258 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 08 09:07:02.010950 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010935 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.011021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010965 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.011021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.010979 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.011426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.011412 2572 policy_none.go:49] "None policy: Start" Jul 08 09:07:02.011472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.011432 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 09:07:02.011472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.011446 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 09:07:02.012364 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.012354 2572 policy_none.go:47] "Start" Jul 08 09:07:02.057675 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.057505 2572 manager.go:337] "Starting Device Plugin manager" Jul 08 09:07:02.057675 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.057541 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 09:07:02.057675 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.057550 2572 server.go:87] "Starting device plugin registration server" Jul 08 09:07:02.057866 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.057804 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 09:07:02.057866 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.057817 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 09:07:02.058033 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.057914 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 09:07:02.058075 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.058048 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 09:07:02.058075 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.058058 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 09:07:02.058548 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.058529 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 09:07:02.058626 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.058576 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.158349 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.158311 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.159596 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.159263 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.159596 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.159286 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.159596 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.159295 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.159596 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.159319 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.167744 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.167712 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 09:07:02.169015 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.168991 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 09:07:02.169109 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.169026 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 09:07:02.169109 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.169047 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 09:07:02.169109 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.169081 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 09:07:02.169268 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.169254 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.169306 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.169278 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-252.ec2.internal\": node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.172117 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.172100 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:07:02.228411 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.228373 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.269471 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.269438 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal"] Jul 08 09:07:02.269555 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.269521 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.271186 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.271169 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.271278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.271203 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.271278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.271215 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.272485 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.272473 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.272603 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.272589 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.272652 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.272629 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.273165 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.273149 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.273257 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.273182 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.273257 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.273192 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.273618 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.273602 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.273669 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.273633 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.273669 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.273648 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.274819 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.274801 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.274881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.274840 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 09:07:02.275466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.275444 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientMemory" Jul 08 09:07:02.275544 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.275470 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 09:07:02.275544 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.275481 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeHasSufficientPID" Jul 08 09:07:02.292625 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.292596 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a79d661b46a3750ad60916d8cdd2a4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal\" (UID: \"6a79d661b46a3750ad60916d8cdd2a4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.292719 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.292628 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a79d661b46a3750ad60916d8cdd2a4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal\" (UID: \"6a79d661b46a3750ad60916d8cdd2a4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.292719 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.292646 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5a24e32bf99f9fe88f14a0d838a9684d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-252.ec2.internal\" (UID: \"5a24e32bf99f9fe88f14a0d838a9684d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.299575 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.299550 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-252.ec2.internal\" not found" node="ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.304078 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.304061 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-252.ec2.internal\" not found" node="ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.328976 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.328912 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.393176 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.393145 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a79d661b46a3750ad60916d8cdd2a4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal\" (UID: \"6a79d661b46a3750ad60916d8cdd2a4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.393176 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.393180 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a79d661b46a3750ad60916d8cdd2a4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal\" (UID: \"6a79d661b46a3750ad60916d8cdd2a4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.393371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.393205 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5a24e32bf99f9fe88f14a0d838a9684d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-252.ec2.internal\" (UID: \"5a24e32bf99f9fe88f14a0d838a9684d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.393371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.393243 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a79d661b46a3750ad60916d8cdd2a4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal\" (UID: \"6a79d661b46a3750ad60916d8cdd2a4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.393371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.393256 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5a24e32bf99f9fe88f14a0d838a9684d-config\") pod \"kube-apiserver-proxy-ip-10-0-140-252.ec2.internal\" (UID: \"5a24e32bf99f9fe88f14a0d838a9684d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.393371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.393244 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a79d661b46a3750ad60916d8cdd2a4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal\" (UID: \"6a79d661b46a3750ad60916d8cdd2a4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.429548 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.429519 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.529755 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.529706 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.602234 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.602157 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.606936 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.606886 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.630025 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.629999 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.730563 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.730529 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.831131 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:02.831091 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-252.ec2.internal\" not found" Jul 08 09:07:02.923838 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.923819 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:07:02.932140 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.932122 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 09:07:02.932232 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.932219 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 09:07:02.932292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.932271 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 09:07:02.973335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.973307 2572 apiserver.go:52] "Watching apiserver" Jul 08 09:07:02.990776 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.990749 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 09:07:02.990925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.990788 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" Jul 08 09:07:02.994968 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.994814 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 09:07:02.996038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.996013 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-8299v","openshift-cluster-node-tuning-operator/tuned-4j9h7","openshift-image-registry/node-ca-crgh2","openshift-multus/multus-8whtj","openshift-multus/multus-additional-cni-plugins-rtxtw","openshift-network-diagnostics/network-check-target-84spb","kube-system/global-pull-secret-syncer-sl4vp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm","openshift-dns/node-resolver-486wp","openshift-multus/network-metrics-daemon-fgcz8","openshift-network-operator/iptables-alerter-4nrgx","openshift-ovn-kubernetes/ovnkube-node-5xmbx"] Jul 08 09:07:02.997683 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.997666 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:02.999286 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:02.999269 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.000288 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.000273 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.000884 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.000869 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 09:07:03.001223 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.001205 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sdztk\"" Jul 08 09:07:03.001337 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.001324 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 09:07:03.001574 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.001561 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wg45b\"" Jul 08 09:07:03.001679 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.001663 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.001741 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.001725 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.001791 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.001764 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.002316 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002299 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.002463 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002447 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 09:07:03.002591 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002569 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 09:02:01 +0000 UTC" deadline="2027-12-23 12:23:43.195571328 +0000 UTC" Jul 08 09:07:03.002635 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002594 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12795h16m40.192981036s" Jul 08 09:07:03.002690 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002677 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.002727 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002716 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mt7gs\"" Jul 08 09:07:03.002843 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.002825 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.004306 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.004292 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:03.004380 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.004362 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:03.004679 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.004663 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 09:07:03.004748 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.004737 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" Jul 08 09:07:03.005049 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.005032 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.005099 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.005040 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.005186 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.005172 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b2d89\"" Jul 08 09:07:03.005235 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.005194 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.005285 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.005255 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:03.005285 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.005279 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 09:07:03.005417 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.005401 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 09:07:03.006501 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.006485 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.007642 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.007630 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.008513 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.008498 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 09:07:03.008550 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.008523 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 09:07:03.008594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.008529 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ps99l\"" Jul 08 09:07:03.008798 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.008783 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 09:07:03.008859 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.008832 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.008949 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.008933 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ch98w\"" Jul 08 09:07:03.009032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.009017 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 09:07:03.009080 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.009033 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.009116 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.009093 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:03.009202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.009188 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.009999 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.009982 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.010128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.010114 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4x8v9\"" Jul 08 09:07:03.010302 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.010291 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.010423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.010407 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.011509 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.011493 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal"] Jul 08 09:07:03.011581 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.011571 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.012526 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.012507 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.012608 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.012582 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 09:07:03.012701 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.012688 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zn6wp\"" Jul 08 09:07:03.012774 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.012761 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.014734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.014718 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 09:07:03.015462 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015447 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 09:07:03.015530 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015491 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mb8ts\"" Jul 08 09:07:03.015583 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015546 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal"] Jul 08 09:07:03.015718 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015700 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 09:07:03.015718 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015707 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 09:07:03.015861 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015791 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 09:07:03.015979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.015965 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 09:07:03.016945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.016930 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 09:07:03.034113 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.034095 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wmsk8" Jul 08 09:07:03.046280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.046265 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wmsk8" Jul 08 09:07:03.092168 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.092152 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 09:07:03.095945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.095923 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:03.096038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.095953 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbvm4\" (UniqueName: \"kubernetes.io/projected/e77a0967-6023-4cc7-98da-290dcea1391a-kube-api-access-hbvm4\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.096038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.095971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.096038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.095991 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-kubernetes\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096019 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/862d5d42-03dd-41be-a8c4-923c31d4ba80-kubelet-config\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.096190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096047 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df-agent-certs\") pod \"konnectivity-agent-8299v\" (UID: \"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df\") " pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.096190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096071 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-modprobe-d\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096100 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsq8c\" (UniqueName: \"kubernetes.io/projected/ad916d36-3a34-4b04-96dd-d831d52ea3c6-kube-api-access-nsq8c\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.096190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096160 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-daemon-config\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096309 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-cni-netd\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.096309 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096246 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f188292-1eae-472d-885b-9f5462a18081-host-slash\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.096309 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096277 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysconfig\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096309 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096300 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-systemd\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096324 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-cni-bin\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096349 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-cnibin\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.096437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096389 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.096437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096414 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xpb9\" (UniqueName: \"kubernetes.io/projected/2b2b7a91-be48-482c-8469-e35e0c9b4549-kube-api-access-5xpb9\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096431 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-device-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096446 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096465 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e77a0967-6023-4cc7-98da-290dcea1391a-hosts-file\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096482 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-socket-dir-parent\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096497 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-cni-bin\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096511 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9v2q\" (UniqueName: \"kubernetes.io/projected/8f188292-1eae-472d-885b-9f5462a18081-kube-api-access-z9v2q\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df-konnectivity-ca\") pod \"konnectivity-agent-8299v\" (UID: \"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df\") " pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096560 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysctl-d\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096573 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-run\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096592 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-os-release\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096615 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-registration-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096641 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-systemd-units\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096656 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-tuned\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096701 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f188292-1eae-472d-885b-9f5462a18081-iptables-alerter-script\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096737 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-cni-multus\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096781 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-systemd\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096807 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-sys\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-var-lib-kubelet\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096873 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-cni-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096923 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-cnibin\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096949 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-system-cni-dir\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.096973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.096974 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnj5h\" (UniqueName: \"kubernetes.io/projected/069db77f-888a-44c4-99b4-08d9bba7cddc-kube-api-access-bnj5h\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097000 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-socket-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097022 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-slash\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097043 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-tmp\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097069 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e77a0967-6023-4cc7-98da-290dcea1391a-tmp-dir\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-system-cni-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097106 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-etc-kubernetes\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097131 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097148 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-run-netns\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097165 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-node-log\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097183 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysctl-conf\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097259 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-host\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097278 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovn-node-metrics-cert\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097315 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv47l\" (UniqueName: \"kubernetes.io/projected/7633f012-7cbf-4d8f-8ef5-31c9269099b0-kube-api-access-jv47l\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097336 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-k8s-cni-cncf-io\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097351 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-conf-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097366 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-multus-certs\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097396 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/862d5d42-03dd-41be-a8c4-923c31d4ba80-dbus\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097415 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-sys-fs\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097430 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcgv6\" (UniqueName: \"kubernetes.io/projected/8c3d6d5b-04ee-420f-a078-8e63b08436e4-kube-api-access-tcgv6\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097458 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-kubelet\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097474 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-log-socket\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097500 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-os-release\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097551 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097592 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4sk9\" (UniqueName: \"kubernetes.io/projected/d0ea32a3-b526-4703-ac99-eddda38f2c87-kube-api-access-g4sk9\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097630 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097665 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-var-lib-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097708 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad916d36-3a34-4b04-96dd-d831d52ea3c6-serviceca\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097730 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-netns\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097751 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.097956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097774 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-ovn\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097799 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtlst\" (UniqueName: \"kubernetes.io/projected/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-kube-api-access-mtlst\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097825 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b2b7a91-be48-482c-8469-e35e0c9b4549-cni-binary-copy\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097861 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-env-overrides\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097874 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovnkube-script-lib\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097890 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097921 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad916d36-3a34-4b04-96dd-d831d52ea3c6-host\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097938 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-lib-modules\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097952 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-kubelet\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097966 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-hostroot\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.097987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-etc-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.098504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.098022 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovnkube-config\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.111865 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.111843 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:07:03.198943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198857 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-cni-bin\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.198943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-cnibin\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.198943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198921 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.198943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198937 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xpb9\" (UniqueName: \"kubernetes.io/projected/2b2b7a91-be48-482c-8469-e35e0c9b4549-kube-api-access-5xpb9\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198957 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-device-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198970 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-cni-bin\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198981 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198974 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-cnibin\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.198978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199029 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-device-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199047 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e77a0967-6023-4cc7-98da-290dcea1391a-hosts-file\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199066 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-etc-selinux\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199098 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e77a0967-6023-4cc7-98da-290dcea1391a-hosts-file\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199102 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-socket-dir-parent\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199069 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-socket-dir-parent\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-cni-bin\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9v2q\" (UniqueName: \"kubernetes.io/projected/8f188292-1eae-472d-885b-9f5462a18081-kube-api-access-z9v2q\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df-konnectivity-ca\") pod \"konnectivity-agent-8299v\" (UID: \"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df\") " pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.199214 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199190 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysctl-d\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199232 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-cni-bin\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199277 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-run\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199295 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-os-release\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199310 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-registration-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199325 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-systemd-units\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-tuned\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199351 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysctl-d\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199389 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-systemd-units\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199392 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-os-release\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199419 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-run\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199449 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f188292-1eae-472d-885b-9f5462a18081-iptables-alerter-script\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-cni-multus\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199484 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-systemd\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199498 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-sys\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-var-lib-kubelet\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199541 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-cni-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199551 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-systemd\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.199807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199564 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-cnibin\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199583 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-registration-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199585 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-sys\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199593 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-var-lib-kubelet\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199622 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-system-cni-dir\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199645 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnj5h\" (UniqueName: \"kubernetes.io/projected/069db77f-888a-44c4-99b4-08d9bba7cddc-kube-api-access-bnj5h\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199652 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-cni-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-socket-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199675 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-system-cni-dir\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199645 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-cnibin\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199676 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-slash\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199714 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-slash\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199713 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-cni-multus\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199741 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-tmp\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199764 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df-konnectivity-ca\") pod \"konnectivity-agent-8299v\" (UID: \"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df\") " pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e77a0967-6023-4cc7-98da-290dcea1391a-tmp-dir\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199796 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199802 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-system-cni-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.200660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199775 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-socket-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-etc-kubernetes\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199866 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-system-cni-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199915 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199921 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-etc-kubernetes\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-run-netns\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199964 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-node-log\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199990 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199997 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.199992 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-run-netns\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200022 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-node-log\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysctl-conf\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200040 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e77a0967-6023-4cc7-98da-290dcea1391a-tmp-dir\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200050 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-host\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200082 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-host\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200092 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.201504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200123 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovn-node-metrics-cert\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200129 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysctl-conf\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200119 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f188292-1eae-472d-885b-9f5462a18081-iptables-alerter-script\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200150 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv47l\" (UniqueName: \"kubernetes.io/projected/7633f012-7cbf-4d8f-8ef5-31c9269099b0-kube-api-access-jv47l\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200179 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-k8s-cni-cncf-io\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200202 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-conf-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200206 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/069db77f-888a-44c4-99b4-08d9bba7cddc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-multus-certs\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200246 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/862d5d42-03dd-41be-a8c4-923c31d4ba80-dbus\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200250 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-k8s-cni-cncf-io\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-conf-dir\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-sys-fs\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200294 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-multus-certs\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcgv6\" (UniqueName: \"kubernetes.io/projected/8c3d6d5b-04ee-420f-a078-8e63b08436e4-kube-api-access-tcgv6\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200339 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-kubelet\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200343 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0ea32a3-b526-4703-ac99-eddda38f2c87-sys-fs\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200344 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/862d5d42-03dd-41be-a8c4-923c31d4ba80-dbus\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.202318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200375 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-kubelet\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200441 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-log-socket\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-os-release\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4sk9\" (UniqueName: \"kubernetes.io/projected/d0ea32a3-b526-4703-ac99-eddda38f2c87-kube-api-access-g4sk9\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200530 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200550 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-var-lib-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200574 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-os-release\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200604 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-var-lib-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200631 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-log-socket\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.200703 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200726 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad916d36-3a34-4b04-96dd-d831d52ea3c6-serviceca\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200752 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-netns\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.200706 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200785 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-run-netns\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.200794 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:03.700759248 +0000 UTC m=+2.055770808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-ovn\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.203053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200883 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtlst\" (UniqueName: \"kubernetes.io/projected/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-kube-api-access-mtlst\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200923 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b2b7a91-be48-482c-8469-e35e0c9b4549-cni-binary-copy\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200945 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.200992 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-run-ovn\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-env-overrides\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201085 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovnkube-script-lib\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201109 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad916d36-3a34-4b04-96dd-d831d52ea3c6-host\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201146 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad916d36-3a34-4b04-96dd-d831d52ea3c6-serviceca\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-lib-modules\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.201187 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:03.701167241 +0000 UTC m=+2.056178783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201265 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-lib-modules\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201315 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad916d36-3a34-4b04-96dd-d831d52ea3c6-host\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201370 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201488 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201540 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-kubelet\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201568 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-hostroot\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201593 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-etc-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201619 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovnkube-config\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201646 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201673 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbvm4\" (UniqueName: \"kubernetes.io/projected/e77a0967-6023-4cc7-98da-290dcea1391a-kube-api-access-hbvm4\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201703 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-kubernetes\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/862d5d42-03dd-41be-a8c4-923c31d4ba80-kubelet-config\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201790 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df-agent-certs\") pod \"konnectivity-agent-8299v\" (UID: \"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df\") " pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201807 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-host-var-lib-kubelet\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201818 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-modprobe-d\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201848 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsq8c\" (UniqueName: \"kubernetes.io/projected/ad916d36-3a34-4b04-96dd-d831d52ea3c6-kube-api-access-nsq8c\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201855 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b2b7a91-be48-482c-8469-e35e0c9b4549-hostroot\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201853 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-kubernetes\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201645 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-daemon-config\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201923 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-cni-netd\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201755 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b2b7a91-be48-482c-8469-e35e0c9b4549-cni-binary-copy\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.204870 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201998 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-etc-openvswitch\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.201924 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovnkube-script-lib\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202292 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/862d5d42-03dd-41be-a8c4-923c31d4ba80-kubelet-config\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202372 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-modprobe-d\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202375 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b2b7a91-be48-482c-8469-e35e0c9b4549-multus-daemon-config\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202426 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7633f012-7cbf-4d8f-8ef5-31c9269099b0-host-cni-netd\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202426 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f188292-1eae-472d-885b-9f5462a18081-host-slash\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202468 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f188292-1eae-472d-885b-9f5462a18081-host-slash\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202515 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysconfig\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202550 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-systemd\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202626 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-systemd\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202630 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-sysconfig\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.202958 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-env-overrides\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.203024 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/069db77f-888a-44c4-99b4-08d9bba7cddc-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.203113 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovnkube-config\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.203294 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-tmp\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.203468 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7633f012-7cbf-4d8f-8ef5-31c9269099b0-ovn-node-metrics-cert\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.203581 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-etc-tuned\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.205716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.204397 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df-agent-certs\") pod \"konnectivity-agent-8299v\" (UID: \"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df\") " pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.208983 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.208956 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9v2q\" (UniqueName: \"kubernetes.io/projected/8f188292-1eae-472d-885b-9f5462a18081-kube-api-access-z9v2q\") pod \"iptables-alerter-4nrgx\" (UID: \"8f188292-1eae-472d-885b-9f5462a18081\") " pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.209630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.209613 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnj5h\" (UniqueName: \"kubernetes.io/projected/069db77f-888a-44c4-99b4-08d9bba7cddc-kube-api-access-bnj5h\") pod \"multus-additional-cni-plugins-rtxtw\" (UID: \"069db77f-888a-44c4-99b4-08d9bba7cddc\") " pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.213640 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.213616 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsq8c\" (UniqueName: \"kubernetes.io/projected/ad916d36-3a34-4b04-96dd-d831d52ea3c6-kube-api-access-nsq8c\") pod \"node-ca-crgh2\" (UID: \"ad916d36-3a34-4b04-96dd-d831d52ea3c6\") " pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.213742 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.213668 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:03.213742 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.213681 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:03.213742 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.213690 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:03.213956 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.213755 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:07:03.713737851 +0000 UTC m=+2.068749411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:03.213956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.213855 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcgv6\" (UniqueName: \"kubernetes.io/projected/8c3d6d5b-04ee-420f-a078-8e63b08436e4-kube-api-access-tcgv6\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.213956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.213887 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtlst\" (UniqueName: \"kubernetes.io/projected/113e2c95-0dbd-4d0c-b3be-c06a35c47e78-kube-api-access-mtlst\") pod \"tuned-4j9h7\" (UID: \"113e2c95-0dbd-4d0c-b3be-c06a35c47e78\") " pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.214145 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.213957 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv47l\" (UniqueName: \"kubernetes.io/projected/7633f012-7cbf-4d8f-8ef5-31c9269099b0-kube-api-access-jv47l\") pod \"ovnkube-node-5xmbx\" (UID: \"7633f012-7cbf-4d8f-8ef5-31c9269099b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.214145 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.214066 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbvm4\" (UniqueName: \"kubernetes.io/projected/e77a0967-6023-4cc7-98da-290dcea1391a-kube-api-access-hbvm4\") pod \"node-resolver-486wp\" (UID: \"e77a0967-6023-4cc7-98da-290dcea1391a\") " pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.214243 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.214218 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4sk9\" (UniqueName: \"kubernetes.io/projected/d0ea32a3-b526-4703-ac99-eddda38f2c87-kube-api-access-g4sk9\") pod \"aws-ebs-csi-driver-node-c7nkm\" (UID: \"d0ea32a3-b526-4703-ac99-eddda38f2c87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.216631 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.216604 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xpb9\" (UniqueName: \"kubernetes.io/projected/2b2b7a91-be48-482c-8469-e35e0c9b4549-kube-api-access-5xpb9\") pod \"multus-8whtj\" (UID: \"2b2b7a91-be48-482c-8469-e35e0c9b4549\") " pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.238594 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.238568 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a24e32bf99f9fe88f14a0d838a9684d.slice/crio-585cf9ca8e2ec828a40b766a231d7ff2ab79635fb99aa8f31d5b4a25db4e9036 WatchSource:0}: Error finding container 585cf9ca8e2ec828a40b766a231d7ff2ab79635fb99aa8f31d5b4a25db4e9036: Status 404 returned error can't find the container with id 585cf9ca8e2ec828a40b766a231d7ff2ab79635fb99aa8f31d5b4a25db4e9036 Jul 08 09:07:03.242738 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.242725 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:07:03.329539 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.329504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:03.337540 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.337518 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904f4b1b_a4c1_4bcf_bb43_fa4a99ed94df.slice/crio-0407294489e7b387fa98fe776811dc81c29a9bd5fe0e0c9b7ba29b520366e05d WatchSource:0}: Error finding container 0407294489e7b387fa98fe776811dc81c29a9bd5fe0e0c9b7ba29b520366e05d: Status 404 returned error can't find the container with id 0407294489e7b387fa98fe776811dc81c29a9bd5fe0e0c9b7ba29b520366e05d Jul 08 09:07:03.345883 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.345864 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" Jul 08 09:07:03.353003 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.352980 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113e2c95_0dbd_4d0c_b3be_c06a35c47e78.slice/crio-5d82a635ca7c5468e8af0e5d4372e05819bfc27f6e885916c8c36714f77964fe WatchSource:0}: Error finding container 5d82a635ca7c5468e8af0e5d4372e05819bfc27f6e885916c8c36714f77964fe: Status 404 returned error can't find the container with id 5d82a635ca7c5468e8af0e5d4372e05819bfc27f6e885916c8c36714f77964fe Jul 08 09:07:03.368496 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.368476 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-crgh2" Jul 08 09:07:03.372390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.372373 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8whtj" Jul 08 09:07:03.375739 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.375719 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad916d36_3a34_4b04_96dd_d831d52ea3c6.slice/crio-e11ca64035328046c6e8e98e1c4015e765603dae4e8d9334af40433d82c47e6a WatchSource:0}: Error finding container e11ca64035328046c6e8e98e1c4015e765603dae4e8d9334af40433d82c47e6a: Status 404 returned error can't find the container with id e11ca64035328046c6e8e98e1c4015e765603dae4e8d9334af40433d82c47e6a Jul 08 09:07:03.379943 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.379925 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b2b7a91_be48_482c_8469_e35e0c9b4549.slice/crio-70f682166e014091a00b3e39272e498a7a8791df359cf35dd064228fa3a33720 WatchSource:0}: Error finding container 70f682166e014091a00b3e39272e498a7a8791df359cf35dd064228fa3a33720: Status 404 returned error can't find the container with id 70f682166e014091a00b3e39272e498a7a8791df359cf35dd064228fa3a33720 Jul 08 09:07:03.385446 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.385428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" Jul 08 09:07:03.397871 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.397858 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" Jul 08 09:07:03.404165 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.404145 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ea32a3_b526_4703_ac99_eddda38f2c87.slice/crio-8c5d361d3edf8f38c91946857651b9fb76a4e32f111eee0b86a2701e0607b9a9 WatchSource:0}: Error finding container 8c5d361d3edf8f38c91946857651b9fb76a4e32f111eee0b86a2701e0607b9a9: Status 404 returned error can't find the container with id 8c5d361d3edf8f38c91946857651b9fb76a4e32f111eee0b86a2701e0607b9a9 Jul 08 09:07:03.409954 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.409938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-486wp" Jul 08 09:07:03.415839 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.415823 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4nrgx" Jul 08 09:07:03.416062 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.416041 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77a0967_6023_4cc7_98da_290dcea1391a.slice/crio-28b5b522a31c36c7f016c76569a364f3426b935a9f39bdb7b8d4bdcc8913618f WatchSource:0}: Error finding container 28b5b522a31c36c7f016c76569a364f3426b935a9f39bdb7b8d4bdcc8913618f: Status 404 returned error can't find the container with id 28b5b522a31c36c7f016c76569a364f3426b935a9f39bdb7b8d4bdcc8913618f Jul 08 09:07:03.420135 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.420113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:03.423037 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.423019 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f188292_1eae_472d_885b_9f5462a18081.slice/crio-5d73575ca4b7111d6c55e8b333fc5078bc37d1ca48cad4ded7e803d91f4c24d4 WatchSource:0}: Error finding container 5d73575ca4b7111d6c55e8b333fc5078bc37d1ca48cad4ded7e803d91f4c24d4: Status 404 returned error can't find the container with id 5d73575ca4b7111d6c55e8b333fc5078bc37d1ca48cad4ded7e803d91f4c24d4 Jul 08 09:07:03.427572 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.427555 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7633f012_7cbf_4d8f_8ef5_31c9269099b0.slice/crio-6fdc1f516e1fd5a31a3c4d50d724c76822c58604ac44d623b1e23bc4eadec954 WatchSource:0}: Error finding container 6fdc1f516e1fd5a31a3c4d50d724c76822c58604ac44d623b1e23bc4eadec954: Status 404 returned error can't find the container with id 6fdc1f516e1fd5a31a3c4d50d724c76822c58604ac44d623b1e23bc4eadec954 Jul 08 09:07:03.433348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.433333 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:07:03.706711 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.706678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:03.706897 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.706725 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:03.706897 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.706849 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:03.707025 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.706849 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:03.707025 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.706999 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:04.706965069 +0000 UTC m=+3.061976615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:03.707134 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.707052 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:04.707030499 +0000 UTC m=+3.062042055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:03.730750 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:03.730718 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a79d661b46a3750ad60916d8cdd2a4c.slice/crio-d1081e96d7d5aeb4f83ec424548bb78d0189b0b580cfba96c577cfde9ea4b629 WatchSource:0}: Error finding container d1081e96d7d5aeb4f83ec424548bb78d0189b0b580cfba96c577cfde9ea4b629: Status 404 returned error can't find the container with id d1081e96d7d5aeb4f83ec424548bb78d0189b0b580cfba96c577cfde9ea4b629 Jul 08 09:07:03.807405 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.807361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:03.807591 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.807529 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:03.807591 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.807547 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:03.807591 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.807560 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:03.807753 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:03.807620 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:07:04.807602238 +0000 UTC m=+3.162613796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:03.883280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:03.883248 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:07:04.048229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.048124 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 09:02:03 +0000 UTC" deadline="2028-03-14 02:58:35.412190873 +0000 UTC" Jul 08 09:07:04.048229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.048183 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14753h51m31.364012136s" Jul 08 09:07:04.172561 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.172531 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:04.172743 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.172669 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:04.190811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.190759 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerStarted","Data":"3188cc135adaf017f5899023d620b1304ace72d211781f757a0a8c5b9438f44d"} Jul 08 09:07:04.201756 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.201726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8whtj" event={"ID":"2b2b7a91-be48-482c-8469-e35e0c9b4549","Type":"ContainerStarted","Data":"70f682166e014091a00b3e39272e498a7a8791df359cf35dd064228fa3a33720"} Jul 08 09:07:04.210964 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.210896 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" event={"ID":"113e2c95-0dbd-4d0c-b3be-c06a35c47e78","Type":"ContainerStarted","Data":"5d82a635ca7c5468e8af0e5d4372e05819bfc27f6e885916c8c36714f77964fe"} Jul 08 09:07:04.228464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.228379 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"6fdc1f516e1fd5a31a3c4d50d724c76822c58604ac44d623b1e23bc4eadec954"} Jul 08 09:07:04.237506 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.237483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4nrgx" event={"ID":"8f188292-1eae-472d-885b-9f5462a18081","Type":"ContainerStarted","Data":"5d73575ca4b7111d6c55e8b333fc5078bc37d1ca48cad4ded7e803d91f4c24d4"} Jul 08 09:07:04.245464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.245404 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-486wp" event={"ID":"e77a0967-6023-4cc7-98da-290dcea1391a","Type":"ContainerStarted","Data":"28b5b522a31c36c7f016c76569a364f3426b935a9f39bdb7b8d4bdcc8913618f"} Jul 08 09:07:04.252569 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.252546 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-crgh2" event={"ID":"ad916d36-3a34-4b04-96dd-d831d52ea3c6","Type":"ContainerStarted","Data":"e11ca64035328046c6e8e98e1c4015e765603dae4e8d9334af40433d82c47e6a"} Jul 08 09:07:04.268005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.267979 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8299v" event={"ID":"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df","Type":"ContainerStarted","Data":"0407294489e7b387fa98fe776811dc81c29a9bd5fe0e0c9b7ba29b520366e05d"} Jul 08 09:07:04.275122 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.275095 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" event={"ID":"5a24e32bf99f9fe88f14a0d838a9684d","Type":"ContainerStarted","Data":"585cf9ca8e2ec828a40b766a231d7ff2ab79635fb99aa8f31d5b4a25db4e9036"} Jul 08 09:07:04.277265 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.277241 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" event={"ID":"6a79d661b46a3750ad60916d8cdd2a4c","Type":"ContainerStarted","Data":"d1081e96d7d5aeb4f83ec424548bb78d0189b0b580cfba96c577cfde9ea4b629"} Jul 08 09:07:04.282269 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.281964 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" event={"ID":"d0ea32a3-b526-4703-ac99-eddda38f2c87","Type":"ContainerStarted","Data":"8c5d361d3edf8f38c91946857651b9fb76a4e32f111eee0b86a2701e0607b9a9"} Jul 08 09:07:04.715762 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.715729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:04.715973 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.715777 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:04.715973 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.715894 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:04.715973 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.715928 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:04.716131 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.715979 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:06.715959277 +0000 UTC m=+5.070970821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:04.716131 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.715999 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:06.715989182 +0000 UTC m=+5.071000726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:04.817670 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.817638 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:04.817839 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.817803 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:04.817839 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.817825 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:04.817839 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.817838 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:04.818014 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:04.817893 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:07:06.8178744 +0000 UTC m=+5.172885953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:04.998299 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:04.998220 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 09:07:05.048891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:05.048854 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 09:02:03 +0000 UTC" deadline="2028-04-01 16:31:42.339157856 +0000 UTC" Jul 08 09:07:05.048891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:05.048891 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15199h24m37.290272928s" Jul 08 09:07:05.170173 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:05.170140 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:05.170349 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:05.170143 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:05.170349 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:05.170279 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:05.170465 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:05.170358 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:06.172630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:06.172051 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:06.172630 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.172197 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:06.732925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:06.732835 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:06.732925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:06.732886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:06.733164 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.733020 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:06.733164 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.733071 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:06.733164 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.733096 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:10.733076629 +0000 UTC m=+9.088088175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:06.733164 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.733128 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:10.73311098 +0000 UTC m=+9.088122526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:06.834206 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:06.833597 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:06.834206 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.833756 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:06.834206 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.833780 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:06.834206 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.833795 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:06.834206 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:06.833859 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:07:10.833841422 +0000 UTC m=+9.188852964 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:07.170247 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:07.170215 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:07.170426 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:07.170344 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:07.170426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:07.170392 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:07.170535 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:07.170515 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:08.177758 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:08.176829 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:08.177758 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:08.177117 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:09.169601 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:09.169567 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:09.169782 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:09.169605 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:09.169782 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:09.169690 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:09.169885 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:09.169830 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:10.169766 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:10.169519 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:10.170216 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.169892 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:10.764648 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:10.764598 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:10.764849 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:10.764671 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:10.764849 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.764770 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:10.764849 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.764780 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:10.764849 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.764849 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:18.764828158 +0000 UTC m=+17.119839712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:10.765099 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.764872 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:18.764862214 +0000 UTC m=+17.119873788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:10.865560 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:10.865522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:10.865735 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.865671 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:10.865735 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.865696 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:10.865735 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.865707 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:10.865888 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:10.865767 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:07:18.865748902 +0000 UTC m=+17.220760444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:11.169413 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:11.169380 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:11.169750 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:11.169508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:11.169750 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:11.169512 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:11.169750 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:11.169588 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:12.170759 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:12.170727 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:12.171113 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:12.170874 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:13.169350 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:13.169310 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:13.169541 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:13.169363 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:13.169541 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:13.169451 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:13.169659 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:13.169598 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:14.169749 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:14.169609 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:14.170238 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:14.169823 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:15.169825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:15.169795 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:15.170285 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:15.169947 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:15.170285 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:15.169955 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:15.170285 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:15.170032 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:15.321445 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:15.321190 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" event={"ID":"5a24e32bf99f9fe88f14a0d838a9684d","Type":"ContainerStarted","Data":"f079d8926a4d04da2c1c7f2712f1b7e8f007a37d8d542bd85e7defd09c182327"} Jul 08 09:07:15.337468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:15.337406 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" event={"ID":"113e2c95-0dbd-4d0c-b3be-c06a35c47e78","Type":"ContainerStarted","Data":"70e2299d74bf7b64a3e1936bc81fc3c875c546b8c9b944915360857ba456bdf8"} Jul 08 09:07:15.360188 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:15.360135 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-252.ec2.internal" podStartSLOduration=12.360114301 podStartE2EDuration="12.360114301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:15.339680337 +0000 UTC m=+13.694691900" watchObservedRunningTime="2026-07-08 09:07:15.360114301 +0000 UTC m=+13.715125867" Jul 08 09:07:16.170215 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.169966 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:16.170617 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:16.170338 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:16.341468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.341432 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4nrgx" event={"ID":"8f188292-1eae-472d-885b-9f5462a18081","Type":"ContainerStarted","Data":"14e2541bdb41048639ed0bae40a4a50a6053470845ddac68628e618085394883"} Jul 08 09:07:16.343134 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.343104 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-486wp" event={"ID":"e77a0967-6023-4cc7-98da-290dcea1391a","Type":"ContainerStarted","Data":"974ebd11ea3def82ef828d1f3e28eec01320d96502574b202ea6290e89666496"} Jul 08 09:07:16.344722 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.344695 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-crgh2" event={"ID":"ad916d36-3a34-4b04-96dd-d831d52ea3c6","Type":"ContainerStarted","Data":"a5349db881e6460b2968f566747f114f6c9b129e6c975df3b73c66fa3b570851"} Jul 08 09:07:16.346387 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.346364 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8299v" event={"ID":"904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df","Type":"ContainerStarted","Data":"d7f2c507a5decc770c9a279d5f3f31bc3f2ab083d2d2d8c3af6a47212f502f49"} Jul 08 09:07:16.347947 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.347918 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" event={"ID":"6a79d661b46a3750ad60916d8cdd2a4c","Type":"ContainerStarted","Data":"936c757e9e4afb07d03c5e71a0b54761014172d7c54961dc156a8b2042ced0e4"} Jul 08 09:07:16.349364 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.349340 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" event={"ID":"d0ea32a3-b526-4703-ac99-eddda38f2c87","Type":"ContainerStarted","Data":"95b23c52ccbe4f886ce810380db87ee060bea3e5c66c20ac5dbf5554df03cd71"} Jul 08 09:07:16.350844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.350823 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerStarted","Data":"a9be5ab50d51be04115e86548bbffabded7f90d8ca68bd85ce22c905507923a2"} Jul 08 09:07:16.357668 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.357625 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4j9h7" podStartSLOduration=3.255030138 podStartE2EDuration="14.357611383s" totalImagesPullingTime="11.102581245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:15.360351371 +0000 UTC m=+13.715362934" watchObservedRunningTime="2026-07-08 09:07:16.357611383 +0000 UTC m=+14.712622944" Jul 08 09:07:16.374657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.374612 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4nrgx" podStartSLOduration=3.363478449 podStartE2EDuration="14.374596049s" totalImagesPullingTime="11.0111176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:16.357724714 +0000 UTC m=+14.712736277" watchObservedRunningTime="2026-07-08 09:07:16.374596049 +0000 UTC m=+14.729607628" Jul 08 09:07:16.392603 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.392552 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-crgh2" podStartSLOduration=3.334266837 podStartE2EDuration="14.392534546s" totalImagesPullingTime="11.058267709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:16.392228863 +0000 UTC m=+14.747240457" watchObservedRunningTime="2026-07-08 09:07:16.392534546 +0000 UTC m=+14.747546111" Jul 08 09:07:16.393101 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.393069 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-486wp" podStartSLOduration=3.406623588 podStartE2EDuration="14.393058731s" totalImagesPullingTime="10.986435143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:16.374722665 +0000 UTC m=+14.729734228" watchObservedRunningTime="2026-07-08 09:07:16.393058731 +0000 UTC m=+14.748070295" Jul 08 09:07:16.453218 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:16.453115 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8299v" podStartSLOduration=3.359054313 podStartE2EDuration="14.453100058s" totalImagesPullingTime="11.094045745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:16.452831398 +0000 UTC m=+14.807842960" watchObservedRunningTime="2026-07-08 09:07:16.453100058 +0000 UTC m=+14.808111620" Jul 08 09:07:17.169366 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:17.169329 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:17.169565 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:17.169329 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:17.169565 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:17.169474 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:17.169679 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:17.169568 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:18.169930 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.169886 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:18.170566 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.170020 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:18.178091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.178067 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:18.178709 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.178682 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:18.355150 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.355043 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:18.355653 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.355625 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8299v" Jul 08 09:07:18.483599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.483519 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:18.574343 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.574298 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:18.829531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.829450 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:18.829531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.829489 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:18.829754 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.829630 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:18.829754 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.829708 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:34.829688071 +0000 UTC m=+33.184699627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:18.829754 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.829636 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:18.829882 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.829775 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:07:34.829763544 +0000 UTC m=+33.184775084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:18.930086 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:18.930042 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:18.930260 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.930203 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:18.930260 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.930228 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:18.930260 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.930241 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:18.930393 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:18.930317 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:07:34.930296558 +0000 UTC m=+33.285308101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:19.169301 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:19.169270 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:19.169494 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:19.169281 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:19.169494 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:19.169394 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:19.169605 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:19.169492 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:19.358786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:19.358754 2572 generic.go:356] "Generic (PLEG): container finished" podID="6a79d661b46a3750ad60916d8cdd2a4c" containerID="936c757e9e4afb07d03c5e71a0b54761014172d7c54961dc156a8b2042ced0e4" exitCode=0 Jul 08 09:07:19.359408 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:19.358831 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" event={"ID":"6a79d661b46a3750ad60916d8cdd2a4c","Type":"ContainerDied","Data":"936c757e9e4afb07d03c5e71a0b54761014172d7c54961dc156a8b2042ced0e4"} Jul 08 09:07:19.360677 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:19.360643 2572 generic.go:356] "Generic (PLEG): container finished" podID="069db77f-888a-44c4-99b4-08d9bba7cddc" containerID="a9be5ab50d51be04115e86548bbffabded7f90d8ca68bd85ce22c905507923a2" exitCode=0 Jul 08 09:07:19.360777 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:19.360690 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerDied","Data":"a9be5ab50d51be04115e86548bbffabded7f90d8ca68bd85ce22c905507923a2"} Jul 08 09:07:20.169578 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:20.169538 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:20.169838 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:20.169691 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:21.169282 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:21.169249 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:21.169282 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:21.169283 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:21.169934 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:21.169385 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:21.169934 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:21.169523 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:22.170693 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:22.170658 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:22.171161 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:22.170792 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:22.784576 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:22.784535 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:23.169943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:23.169894 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:23.170113 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:23.169897 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:23.170113 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:23.170032 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:23.170113 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:23.170096 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:23.898343 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:23.898045 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 09:07:24.070234 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.070121 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T09:07:23.898319278Z","UUID":"fcd673ed-683b-4c23-98fa-697a5a724a07","Handler":null,"Name":"","Endpoint":""} Jul 08 09:07:24.071743 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.071722 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 09:07:24.071814 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.071769 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 09:07:24.170144 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.170115 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:24.170316 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:24.170279 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:24.374895 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.374854 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"93e5502d4dd23d374d849a48d54c5dbf5675a834088a05cc33ed97b0484bd564"} Jul 08 09:07:24.375084 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.374967 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"c39e62d0ec6b0aa3f8fad20e3c8fad9aa3664aad23bc549e0b6ea7ae38325bea"} Jul 08 09:07:24.375084 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.375017 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"9fc062e60e9147ace870d2d83f5158f8658c9cd5502e6c11647eff9f2f666281"} Jul 08 09:07:24.375084 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.375068 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"453a3f46b1c0c633ad152bde72bfdeab4c513faa52af9b3ec5c68ec560f55586"} Jul 08 09:07:24.375255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.375116 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"4aaba88f5b2f7797a9428eae348cc8700df54cdfaf914c8c5a0c4713693ce25e"} Jul 08 09:07:24.375255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.375163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"57612c930542545015246495726bbb55bdfa1cf6e13c1daf676756d4f384de5a"} Jul 08 09:07:24.376951 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.376926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" event={"ID":"6a79d661b46a3750ad60916d8cdd2a4c","Type":"ContainerStarted","Data":"9cd37ab583da881e768d6910d3d4d084d0668e80580dd0046626d78bd375db0c"} Jul 08 09:07:24.378839 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.378807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" event={"ID":"d0ea32a3-b526-4703-ac99-eddda38f2c87","Type":"ContainerStarted","Data":"a758503fa11f14a7e1954c9e77b99890713bab2621beab69357816171374a1c1"} Jul 08 09:07:24.380472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.380448 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8whtj" event={"ID":"2b2b7a91-be48-482c-8469-e35e0c9b4549","Type":"ContainerStarted","Data":"b3e074be4a3b85377e5570ef766d51a64d296d2286758597b7b7787c304b8f32"} Jul 08 09:07:24.424723 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.424680 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-252.ec2.internal" podStartSLOduration=22.42466798 podStartE2EDuration="22.42466798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:24.398299306 +0000 UTC m=+22.753310867" watchObservedRunningTime="2026-07-08 09:07:24.42466798 +0000 UTC m=+22.779679542" Jul 08 09:07:24.424936 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.424861 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8whtj" podStartSLOduration=2.320432763 podStartE2EDuration="22.42485607s" totalImagesPullingTime="20.104423307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:24.424792994 +0000 UTC m=+22.779804536" watchObservedRunningTime="2026-07-08 09:07:24.42485607 +0000 UTC m=+22.779867632" Jul 08 09:07:24.552306 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.552273 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:24.652552 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.652467 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:24.768307 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.768258 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:24.857043 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:24.857012 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:25.172443 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:25.170029 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:25.172443 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:25.170050 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:25.172443 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:25.170185 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:25.172443 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:25.170399 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:26.169521 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:26.169494 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:26.169712 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:26.169638 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:26.389787 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:26.389674 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"28410569881e3866e082bac3eb66b6db3fff489de0a5f178d6d1972f330ae766"} Jul 08 09:07:26.392094 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:26.392065 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" event={"ID":"d0ea32a3-b526-4703-ac99-eddda38f2c87","Type":"ContainerStarted","Data":"ba0120bff025b2ba7c85039a41eb46e5fad2a34259d3d1eab0fdc729f3ec2ec3"} Jul 08 09:07:26.417584 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:26.417525 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c7nkm" podStartSLOduration=2.967642922 podStartE2EDuration="24.417506286s" totalImagesPullingTime="21.449863364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:26.417389466 +0000 UTC m=+24.772401029" watchObservedRunningTime="2026-07-08 09:07:26.417506286 +0000 UTC m=+24.772517849" Jul 08 09:07:26.465452 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:26.465418 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:26.565792 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:26.565761 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:27.170164 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.170137 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:27.170338 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.170180 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:27.170338 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:27.170257 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:27.170448 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:27.170390 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:27.404438 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.404402 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:27.484345 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.484265 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:27.548370 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.548334 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:27.616576 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.616539 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:27.719466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:27.719437 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:28.169686 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:28.169656 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:28.169859 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:28.169809 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:29.169952 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.169758 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:29.170570 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.169812 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:29.170570 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:29.170024 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:29.170570 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:29.170096 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:29.267005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.266914 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:29.368281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.368248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:29.400778 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.400752 2572 generic.go:356] "Generic (PLEG): container finished" podID="069db77f-888a-44c4-99b4-08d9bba7cddc" containerID="702b8bedea95831b4cc19da0c8a8add99d8976c2854a275c06e8b375b4c2606e" exitCode=0 Jul 08 09:07:29.400919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.400830 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerDied","Data":"702b8bedea95831b4cc19da0c8a8add99d8976c2854a275c06e8b375b4c2606e"} Jul 08 09:07:29.404570 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.404549 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" event={"ID":"7633f012-7cbf-4d8f-8ef5-31c9269099b0","Type":"ContainerStarted","Data":"5e6e2c6b061a8fa04b7f85eaafa0a46a1f56b04be40af1908f8746e867f70887"} Jul 08 09:07:29.404831 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.404816 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:29.404921 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.404887 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:29.404996 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.404951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:29.419991 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.419968 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:29.420123 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.420105 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:07:29.467009 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:29.466966 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" podStartSLOduration=7.450822727 podStartE2EDuration="27.466951749s" totalImagesPullingTime="20.016129022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:29.466393683 +0000 UTC m=+27.821405238" watchObservedRunningTime="2026-07-08 09:07:29.466951749 +0000 UTC m=+27.821963310" Jul 08 09:07:30.169691 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.169664 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:30.169972 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:30.169777 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:30.985394 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.985359 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-84spb"] Jul 08 09:07:30.985572 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.985524 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:30.985661 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:30.985636 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:30.991529 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.991487 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fgcz8"] Jul 08 09:07:30.991699 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.991606 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:30.991784 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:30.991702 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:30.992296 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.992271 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sl4vp"] Jul 08 09:07:30.992388 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:30.992368 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:30.992479 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:30.992459 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:31.410743 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:31.410708 2572 generic.go:356] "Generic (PLEG): container finished" podID="069db77f-888a-44c4-99b4-08d9bba7cddc" containerID="a6c12d5bc4824da274dd269a42c4a420ca77e6ad137d298b9315dd17481717f5" exitCode=0 Jul 08 09:07:31.411138 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:31.410762 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerDied","Data":"a6c12d5bc4824da274dd269a42c4a420ca77e6ad137d298b9315dd17481717f5"} Jul 08 09:07:33.169893 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:33.169718 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:33.169893 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:33.169773 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:33.170546 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:33.169781 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:33.170546 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:33.169990 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:33.170546 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:33.170057 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:33.170546 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:33.170151 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:33.417611 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:33.417576 2572 generic.go:356] "Generic (PLEG): container finished" podID="069db77f-888a-44c4-99b4-08d9bba7cddc" containerID="9e0bfb03a539e10472d6b94d3740ef1df58512bb2720af67bb250f281506750d" exitCode=0 Jul 08 09:07:33.417774 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:33.417626 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerDied","Data":"9e0bfb03a539e10472d6b94d3740ef1df58512bb2720af67bb250f281506750d"} Jul 08 09:07:33.749473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:33.749391 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:34.847256 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:34.847224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:34.847256 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:34.847258 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:34.847670 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.847369 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:34.847670 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.847421 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:06.847406957 +0000 UTC m=+65.202418497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 09:07:34.847670 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.847475 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:34.847670 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.847537 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret podName:862d5d42-03dd-41be-a8c4-923c31d4ba80 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:06.847519433 +0000 UTC m=+65.202530985 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret") pod "global-pull-secret-syncer-sl4vp" (UID: "862d5d42-03dd-41be-a8c4-923c31d4ba80") : object "kube-system"/"original-pull-secret" not registered Jul 08 09:07:34.948207 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:34.948172 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:34.948395 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.948358 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 09:07:34.948395 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.948381 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 09:07:34.948395 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.948393 2572 projected.go:196] Error preparing data for projected volume kube-api-access-69z2p for pod openshift-network-diagnostics/network-check-target-84spb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:34.948546 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:34.948462 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p podName:d5bf9ebe-965e-4642-a9b1-53eb56628c6e nodeName:}" failed. No retries permitted until 2026-07-08 09:08:06.948442404 +0000 UTC m=+65.303453964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-69z2p" (UniqueName: "kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p") pod "network-check-target-84spb" (UID: "d5bf9ebe-965e-4642-a9b1-53eb56628c6e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 09:07:35.170050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:35.170013 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:35.170050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:35.170043 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:35.170260 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:35.170013 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:35.170260 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:35.170128 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84spb" podUID="d5bf9ebe-965e-4642-a9b1-53eb56628c6e" Jul 08 09:07:35.170260 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:35.170192 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fgcz8" podUID="8c3d6d5b-04ee-420f-a078-8e63b08436e4" Jul 08 09:07:35.170364 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:35.170257 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sl4vp" podUID="862d5d42-03dd-41be-a8c4-923c31d4ba80" Jul 08 09:07:36.399545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.399513 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-252.ec2.internal" event="NodeReady" Jul 08 09:07:36.400095 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.399644 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 09:07:36.454216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.454186 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zf8b5"] Jul 08 09:07:36.456130 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.456112 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.457678 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.457658 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jfrkc"] Jul 08 09:07:36.458696 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.458677 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qglp2\"" Jul 08 09:07:36.458800 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.458676 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 09:07:36.458800 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.458772 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 09:07:36.459330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.459315 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.462280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.462262 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8jcgn\"" Jul 08 09:07:36.462407 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.462385 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 09:07:36.462539 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.462507 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 09:07:36.462630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.462601 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 09:07:36.467162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.467143 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zf8b5"] Jul 08 09:07:36.470644 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.470621 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jfrkc"] Jul 08 09:07:36.551286 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.551257 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:36.558999 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.558972 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttdh\" (UniqueName: \"kubernetes.io/projected/fa74e311-430b-42f8-bc39-dd43f477309c-kube-api-access-zttdh\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.559126 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.559011 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spftt\" (UniqueName: \"kubernetes.io/projected/b8c50830-ba2b-427f-ad54-786c4d84b3f7-kube-api-access-spftt\") pod \"ingress-canary-jfrkc\" (UID: \"b8c50830-ba2b-427f-ad54-786c4d84b3f7\") " pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.559126 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.559091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa74e311-430b-42f8-bc39-dd43f477309c-config-volume\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.559232 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.559182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8c50830-ba2b-427f-ad54-786c4d84b3f7-cert\") pod \"ingress-canary-jfrkc\" (UID: \"b8c50830-ba2b-427f-ad54-786c4d84b3f7\") " pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.559275 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.559248 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa74e311-430b-42f8-bc39-dd43f477309c-tmp-dir\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.559306 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.559298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.651393 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.651319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:36.660267 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660239 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8c50830-ba2b-427f-ad54-786c4d84b3f7-cert\") pod \"ingress-canary-jfrkc\" (UID: \"b8c50830-ba2b-427f-ad54-786c4d84b3f7\") " pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.660390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa74e311-430b-42f8-bc39-dd43f477309c-tmp-dir\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.660390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660330 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.660390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660355 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttdh\" (UniqueName: \"kubernetes.io/projected/fa74e311-430b-42f8-bc39-dd43f477309c-kube-api-access-zttdh\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.660390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spftt\" (UniqueName: \"kubernetes.io/projected/b8c50830-ba2b-427f-ad54-786c4d84b3f7-kube-api-access-spftt\") pod \"ingress-canary-jfrkc\" (UID: \"b8c50830-ba2b-427f-ad54-786c4d84b3f7\") " pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.660590 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660405 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa74e311-430b-42f8-bc39-dd43f477309c-config-volume\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.660590 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:36.660454 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:36.660590 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:36.660532 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:07:37.160512211 +0000 UTC m=+35.515523755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:07:36.660752 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.660708 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa74e311-430b-42f8-bc39-dd43f477309c-tmp-dir\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.661524 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.661498 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa74e311-430b-42f8-bc39-dd43f477309c-config-volume\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.664629 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.664598 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8c50830-ba2b-427f-ad54-786c4d84b3f7-cert\") pod \"ingress-canary-jfrkc\" (UID: \"b8c50830-ba2b-427f-ad54-786c4d84b3f7\") " pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.671392 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.671373 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spftt\" (UniqueName: \"kubernetes.io/projected/b8c50830-ba2b-427f-ad54-786c4d84b3f7-kube-api-access-spftt\") pod \"ingress-canary-jfrkc\" (UID: \"b8c50830-ba2b-427f-ad54-786c4d84b3f7\") " pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.672934 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.672896 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttdh\" (UniqueName: \"kubernetes.io/projected/fa74e311-430b-42f8-bc39-dd43f477309c-kube-api-access-zttdh\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:36.773224 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.773195 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jfrkc" Jul 08 09:07:36.910489 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:36.910185 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jfrkc"] Jul 08 09:07:36.914060 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:07:36.914034 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c50830_ba2b_427f_ad54_786c4d84b3f7.slice/crio-d5e4f89d8d9d44aa02d404cab177b6ba99736f08242e376da11b8b1a05ef2447 WatchSource:0}: Error finding container d5e4f89d8d9d44aa02d404cab177b6ba99736f08242e376da11b8b1a05ef2447: Status 404 returned error can't find the container with id d5e4f89d8d9d44aa02d404cab177b6ba99736f08242e376da11b8b1a05ef2447 Jul 08 09:07:37.165244 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.165147 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:37.165394 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:37.165313 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:37.165394 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:37.165374 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:07:38.16535921 +0000 UTC m=+36.520370754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:07:37.170248 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.170220 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:07:37.170248 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.170241 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:07:37.170403 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.170261 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:07:37.173017 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.172992 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gctfj\"" Jul 08 09:07:37.173135 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.173072 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 09:07:37.173135 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.173078 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 09:07:37.173789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.173769 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 09:07:37.173864 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.173823 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 09:07:37.173937 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.173887 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6bnkm\"" Jul 08 09:07:37.428660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:37.428566 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jfrkc" event={"ID":"b8c50830-ba2b-427f-ad54-786c4d84b3f7","Type":"ContainerStarted","Data":"d5e4f89d8d9d44aa02d404cab177b6ba99736f08242e376da11b8b1a05ef2447"} Jul 08 09:07:38.172833 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:38.172802 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:38.173014 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:38.172976 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:38.173076 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:38.173032 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:07:40.173016534 +0000 UTC m=+38.528028077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:07:38.859411 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:38.859372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:40.188962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:40.188930 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:40.189352 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:40.189031 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:40.189352 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:40.189085 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:07:44.189072641 +0000 UTC m=+42.544084182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:07:40.509003 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:40.508973 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:41.441359 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:41.441326 2572 generic.go:356] "Generic (PLEG): container finished" podID="069db77f-888a-44c4-99b4-08d9bba7cddc" containerID="d828a860c59fdd50f0203781a9950abc4b2bc54fc2d37a2bd98a83fdd89ba98e" exitCode=0 Jul 08 09:07:41.441742 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:41.441395 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerDied","Data":"d828a860c59fdd50f0203781a9950abc4b2bc54fc2d37a2bd98a83fdd89ba98e"} Jul 08 09:07:41.442923 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:41.442805 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jfrkc" event={"ID":"b8c50830-ba2b-427f-ad54-786c4d84b3f7","Type":"ContainerStarted","Data":"4894681363dcb54f76b3d7d0cd107bc065afce7182df6886411404b3d62f14f9"} Jul 08 09:07:41.499711 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:41.499650 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jfrkc" podStartSLOduration=2.026615799 podStartE2EDuration="5.499630214s" totalImagesPullingTime="3.473014415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:41.498201541 +0000 UTC m=+39.853213102" watchObservedRunningTime="2026-07-08 09:07:41.499630214 +0000 UTC m=+39.854641777" Jul 08 09:07:42.447876 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:42.447695 2572 generic.go:356] "Generic (PLEG): container finished" podID="069db77f-888a-44c4-99b4-08d9bba7cddc" containerID="c73636a50456af53391ffbd70571c521d836284cfdd23537b70977e36de61d91" exitCode=0 Jul 08 09:07:42.448248 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:42.447765 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerDied","Data":"c73636a50456af53391ffbd70571c521d836284cfdd23537b70977e36de61d91"} Jul 08 09:07:43.452989 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.452953 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" event={"ID":"069db77f-888a-44c4-99b4-08d9bba7cddc","Type":"ContainerStarted","Data":"351cb46afa09f18fbd761f2b0b123d4200e5e71823de1583f0f7356a01e128e8"} Jul 08 09:07:43.480242 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.480192 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rtxtw" podStartSLOduration=11.432993486 podStartE2EDuration="41.480179104s" totalImagesPullingTime="30.047185618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:07:43.478873624 +0000 UTC m=+41.833885186" watchObservedRunningTime="2026-07-08 09:07:43.480179104 +0000 UTC m=+41.835190666" Jul 08 09:07:43.606630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.606585 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:43.709355 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.709248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:43.810134 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.810074 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:43.910558 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.910521 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:43.990146 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:43.990067 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:44.216120 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:44.216087 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:44.216285 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:44.216230 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:44.216328 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:44.216302 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:07:52.21628556 +0000 UTC m=+50.571297103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:07:44.560055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:44.559990 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:44.628070 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:44.628017 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:45.008650 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:45.008618 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:45.088639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:45.088589 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:48.257272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:48.257221 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:07:52.275401 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:07:52.275365 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:07:52.275777 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:52.275517 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:07:52.275777 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:07:52.275577 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:08:08.275560214 +0000 UTC m=+66.630571754 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:08:01.421612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:01.421581 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xmbx" Jul 08 09:08:01.652933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:01.652880 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:01.813712 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:01.813620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:06.159011 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.158977 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:06.260384 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.260344 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:06.870198 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.870162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:08:06.870198 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.870201 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:08:06.873734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.873713 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 09:08:06.874667 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.874650 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 09:08:06.880678 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:06.880665 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 09:08:06.880744 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:06.880732 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs podName:8c3d6d5b-04ee-420f-a078-8e63b08436e4 nodeName:}" failed. No retries permitted until 2026-07-08 09:09:10.880713501 +0000 UTC m=+129.235725044 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs") pod "network-metrics-daemon-fgcz8" (UID: "8c3d6d5b-04ee-420f-a078-8e63b08436e4") : secret "metrics-daemon-secret" not found Jul 08 09:08:06.884475 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.884448 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/862d5d42-03dd-41be-a8c4-923c31d4ba80-original-pull-secret\") pod \"global-pull-secret-syncer-sl4vp\" (UID: \"862d5d42-03dd-41be-a8c4-923c31d4ba80\") " pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:08:06.895559 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.895541 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sl4vp" Jul 08 09:08:06.971151 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.971066 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:08:06.974255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.974169 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 09:08:06.984389 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.984364 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 09:08:06.994205 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:06.994175 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69z2p\" (UniqueName: \"kubernetes.io/projected/d5bf9ebe-965e-4642-a9b1-53eb56628c6e-kube-api-access-69z2p\") pod \"network-check-target-84spb\" (UID: \"d5bf9ebe-965e-4642-a9b1-53eb56628c6e\") " pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:08:07.021897 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:07.021871 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sl4vp"] Jul 08 09:08:07.025864 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:07.025835 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod862d5d42_03dd_41be_a8c4_923c31d4ba80.slice/crio-39ff88f930e620acf4aeed17321ad6046e4524f0e6babf1fb4bdd9200d6c133e WatchSource:0}: Error finding container 39ff88f930e620acf4aeed17321ad6046e4524f0e6babf1fb4bdd9200d6c133e: Status 404 returned error can't find the container with id 39ff88f930e620acf4aeed17321ad6046e4524f0e6babf1fb4bdd9200d6c133e Jul 08 09:08:07.194692 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:07.194613 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gctfj\"" Jul 08 09:08:07.200562 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:07.200547 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:08:07.321843 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:07.321815 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-84spb"] Jul 08 09:08:07.325123 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:07.325088 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5bf9ebe_965e_4642_a9b1_53eb56628c6e.slice/crio-1be2ad11fa1fe37e2fe0a8b6a4d42a38360f7c6037e9030e7531fdce467eee6f WatchSource:0}: Error finding container 1be2ad11fa1fe37e2fe0a8b6a4d42a38360f7c6037e9030e7531fdce467eee6f: Status 404 returned error can't find the container with id 1be2ad11fa1fe37e2fe0a8b6a4d42a38360f7c6037e9030e7531fdce467eee6f Jul 08 09:08:07.509999 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:07.509885 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sl4vp" event={"ID":"862d5d42-03dd-41be-a8c4-923c31d4ba80","Type":"ContainerStarted","Data":"39ff88f930e620acf4aeed17321ad6046e4524f0e6babf1fb4bdd9200d6c133e"} Jul 08 09:08:07.511005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:07.510986 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-84spb" event={"ID":"d5bf9ebe-965e-4642-a9b1-53eb56628c6e","Type":"ContainerStarted","Data":"1be2ad11fa1fe37e2fe0a8b6a4d42a38360f7c6037e9030e7531fdce467eee6f"} Jul 08 09:08:08.279193 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:08.279152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:08:08.279609 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:08.279323 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 09:08:08.279609 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:08.279395 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls podName:fa74e311-430b-42f8-bc39-dd43f477309c nodeName:}" failed. No retries permitted until 2026-07-08 09:08:40.279378062 +0000 UTC m=+98.634389602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls") pod "dns-default-zf8b5" (UID: "fa74e311-430b-42f8-bc39-dd43f477309c") : secret "dns-default-metrics-tls" not found Jul 08 09:08:10.751270 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.751199 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn"] Jul 08 09:08:10.782894 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.782773 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn"] Jul 08 09:08:10.783078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.782916 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" Jul 08 09:08:10.786393 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.786370 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sfw75\"" Jul 08 09:08:10.786535 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.786422 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 08 09:08:10.788136 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.788100 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:08:10.797839 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.797815 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c245p\" (UniqueName: \"kubernetes.io/projected/42d8150b-fb0b-4a6e-bae3-0db02436b2a5-kube-api-access-c245p\") pod \"volume-data-source-validator-5d456fdbcd-5svmn\" (UID: \"42d8150b-fb0b-4a6e-bae3-0db02436b2a5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" Jul 08 09:08:10.838178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.838144 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd"] Jul 08 09:08:10.859164 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.859131 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd"] Jul 08 09:08:10.859300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.859239 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:10.865370 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.865346 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 09:08:10.865480 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.865387 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 09:08:10.865480 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.865400 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 09:08:10.865480 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.865346 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:08:10.865778 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.865758 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8nv2d\"" Jul 08 09:08:10.898667 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.898641 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c245p\" (UniqueName: \"kubernetes.io/projected/42d8150b-fb0b-4a6e-bae3-0db02436b2a5-kube-api-access-c245p\") pod \"volume-data-source-validator-5d456fdbcd-5svmn\" (UID: \"42d8150b-fb0b-4a6e-bae3-0db02436b2a5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" Jul 08 09:08:10.898771 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.898676 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrtzn\" (UniqueName: \"kubernetes.io/projected/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-kube-api-access-zrtzn\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:10.898771 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.898713 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:10.898885 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.898796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:10.910234 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.910192 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c245p\" (UniqueName: \"kubernetes.io/projected/42d8150b-fb0b-4a6e-bae3-0db02436b2a5-kube-api-access-c245p\") pod \"volume-data-source-validator-5d456fdbcd-5svmn\" (UID: \"42d8150b-fb0b-4a6e-bae3-0db02436b2a5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" Jul 08 09:08:10.936247 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.936219 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv"] Jul 08 09:08:10.949925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.949889 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86b94ddf7f-xzhhk"] Jul 08 09:08:10.950059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.950002 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:10.951630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.951604 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:10.954535 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.954514 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 09:08:10.954632 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.954562 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 09:08:10.955547 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.955529 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:08:10.955547 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.955541 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 09:08:10.956308 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.956144 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-p9kzp\"" Jul 08 09:08:10.965983 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.965965 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv"] Jul 08 09:08:10.966095 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.966065 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:10.968868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.968613 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zkvd5\"" Jul 08 09:08:10.968868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.968631 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 09:08:10.968868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.968725 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 09:08:10.969371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.969218 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 09:08:10.975470 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.975443 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 09:08:10.982227 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.982202 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86b94ddf7f-xzhhk"] Jul 08 09:08:10.999717 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999693 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhdbc\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-kube-api-access-lhdbc\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:10.999823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999745 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:10.999823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999772 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f785069b-d367-48c5-bfe2-53cc471b0142-ca-trust-extracted\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:10.999823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999797 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-image-registry-private-configuration\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.000002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999842 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9blrw\" (UniqueName: \"kubernetes.io/projected/9f6e3800-8e80-411b-9506-d6f9f4042f0b-kube-api-access-9blrw\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.000002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999868 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-installation-pull-secrets\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.000002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999919 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrtzn\" (UniqueName: \"kubernetes.io/projected/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-kube-api-access-zrtzn\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.000002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999948 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f6e3800-8e80-411b-9506-d6f9f4042f0b-config\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.000002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:10.999999 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-trusted-ca\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.000216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.000016 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-bound-sa-token\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.000216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.000058 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6e3800-8e80-411b-9506-d6f9f4042f0b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.000216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.000090 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-registry-certificates\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.000216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.000124 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.000216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.000211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.000862 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.000839 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.002636 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.002576 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.013742 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.013723 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrtzn\" (UniqueName: \"kubernetes.io/projected/048aa6fd-5839-43dd-a471-1f58a7ab8fd4-kube-api-access-zrtzn\") pod \"kube-storage-version-migrator-operator-846bf668f7-qpjjd\" (UID: \"048aa6fd-5839-43dd-a471-1f58a7ab8fd4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.058913 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.058866 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:11.093961 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.093937 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" Jul 08 09:08:11.101375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101351 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.101507 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f785069b-d367-48c5-bfe2-53cc471b0142-ca-trust-extracted\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.101507 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101410 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-image-registry-private-configuration\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.101624 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:11.101502 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:11.101624 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:11.101521 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86b94ddf7f-xzhhk: secret "image-registry-tls" not found Jul 08 09:08:11.101624 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101549 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9blrw\" (UniqueName: \"kubernetes.io/projected/9f6e3800-8e80-411b-9506-d6f9f4042f0b-kube-api-access-9blrw\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.101624 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:11.101593 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls podName:f785069b-d367-48c5-bfe2-53cc471b0142 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:11.601572748 +0000 UTC m=+69.956584291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls") pod "image-registry-86b94ddf7f-xzhhk" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142") : secret "image-registry-tls" not found Jul 08 09:08:11.101892 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101629 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-installation-pull-secrets\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.101892 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101673 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f6e3800-8e80-411b-9506-d6f9f4042f0b-config\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.101892 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101718 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-trusted-ca\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.101892 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101743 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-bound-sa-token\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.102208 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.102185 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f785069b-d367-48c5-bfe2-53cc471b0142-ca-trust-extracted\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.102599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.101770 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6e3800-8e80-411b-9506-d6f9f4042f0b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.102702 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.102641 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-registry-certificates\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.103313 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.103262 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-registry-certificates\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.108451 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.103505 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhdbc\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-kube-api-access-lhdbc\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.108451 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.104280 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-trusted-ca\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.108451 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.104843 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-installation-pull-secrets\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.108451 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.107484 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6e3800-8e80-411b-9506-d6f9f4042f0b-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.108451 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.108187 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-image-registry-private-configuration\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.109271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.109250 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f6e3800-8e80-411b-9506-d6f9f4042f0b-config\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.114353 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.114333 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-bound-sa-token\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.114660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.114638 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9blrw\" (UniqueName: \"kubernetes.io/projected/9f6e3800-8e80-411b-9506-d6f9f4042f0b-kube-api-access-9blrw\") pod \"service-ca-operator-5d46d9ffd4-kk6rv\" (UID: \"9f6e3800-8e80-411b-9506-d6f9f4042f0b\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.115233 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.115204 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhdbc\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-kube-api-access-lhdbc\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.167439 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.167413 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" Jul 08 09:08:11.262440 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.262363 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" Jul 08 09:08:11.607529 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:11.607495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:11.607710 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:11.607672 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:11.607710 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:11.607695 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86b94ddf7f-xzhhk: secret "image-registry-tls" not found Jul 08 09:08:11.607830 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:11.607769 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls podName:f785069b-d367-48c5-bfe2-53cc471b0142 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:12.607747043 +0000 UTC m=+70.962758605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls") pod "image-registry-86b94ddf7f-xzhhk" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142") : secret "image-registry-tls" not found Jul 08 09:08:12.484691 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.484662 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv"] Jul 08 09:08:12.487808 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:12.487777 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6e3800_8e80_411b_9506_d6f9f4042f0b.slice/crio-d97c98fc305ec2e24058e4587a1326738c2b5c1c1672cab5e4bbb9ed91195c4b WatchSource:0}: Error finding container d97c98fc305ec2e24058e4587a1326738c2b5c1c1672cab5e4bbb9ed91195c4b: Status 404 returned error can't find the container with id d97c98fc305ec2e24058e4587a1326738c2b5c1c1672cab5e4bbb9ed91195c4b Jul 08 09:08:12.526202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.526171 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-84spb" event={"ID":"d5bf9ebe-965e-4642-a9b1-53eb56628c6e","Type":"ContainerStarted","Data":"0dda183bc33c2bf75900b465782e0416c106288410ebc4086822628baf82524c"} Jul 08 09:08:12.526324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.526304 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:08:12.527679 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.527613 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" event={"ID":"9f6e3800-8e80-411b-9506-d6f9f4042f0b","Type":"ContainerStarted","Data":"d97c98fc305ec2e24058e4587a1326738c2b5c1c1672cab5e4bbb9ed91195c4b"} Jul 08 09:08:12.529979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.529955 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sl4vp" event={"ID":"862d5d42-03dd-41be-a8c4-923c31d4ba80","Type":"ContainerStarted","Data":"a278c74f580f74d72ea1da636b6515c6769a5d33f1433e7a445ecb35fa39f14d"} Jul 08 09:08:12.545874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.545821 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-84spb" podStartSLOduration=65.541400997 podStartE2EDuration="1m10.545807567s" totalImagesPullingTime="5.00440657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:12.544271336 +0000 UTC m=+70.899282899" watchObservedRunningTime="2026-07-08 09:08:12.545807567 +0000 UTC m=+70.900819128" Jul 08 09:08:12.564869 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.564822 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sl4vp" podStartSLOduration=65.247909599 podStartE2EDuration="1m10.564809144s" totalImagesPullingTime="5.316899545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:12.56475818 +0000 UTC m=+70.919769743" watchObservedRunningTime="2026-07-08 09:08:12.564809144 +0000 UTC m=+70.919820684" Jul 08 09:08:12.614773 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.614744 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:12.614974 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:12.614952 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:12.615024 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:12.614976 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86b94ddf7f-xzhhk: secret "image-registry-tls" not found Jul 08 09:08:12.615065 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:12.615052 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls podName:f785069b-d367-48c5-bfe2-53cc471b0142 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:14.615030592 +0000 UTC m=+72.970042164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls") pod "image-registry-86b94ddf7f-xzhhk" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142") : secret "image-registry-tls" not found Jul 08 09:08:12.705117 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.705095 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd"] Jul 08 09:08:12.707182 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:12.707157 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048aa6fd_5839_43dd_a471_1f58a7ab8fd4.slice/crio-afabd35d1288062b519967b16166334074a0c689d9daf5aee8283e59b037b78a WatchSource:0}: Error finding container afabd35d1288062b519967b16166334074a0c689d9daf5aee8283e59b037b78a: Status 404 returned error can't find the container with id afabd35d1288062b519967b16166334074a0c689d9daf5aee8283e59b037b78a Jul 08 09:08:12.710162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:12.710137 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn"] Jul 08 09:08:12.712964 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:12.712940 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d8150b_fb0b_4a6e_bae3_0db02436b2a5.slice/crio-18e2d1fc3695b3eebf1383ec6eda2ec9b85d75e903119dda22f284867799de05 WatchSource:0}: Error finding container 18e2d1fc3695b3eebf1383ec6eda2ec9b85d75e903119dda22f284867799de05: Status 404 returned error can't find the container with id 18e2d1fc3695b3eebf1383ec6eda2ec9b85d75e903119dda22f284867799de05 Jul 08 09:08:13.535041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:13.534953 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" event={"ID":"048aa6fd-5839-43dd-a471-1f58a7ab8fd4","Type":"ContainerStarted","Data":"afabd35d1288062b519967b16166334074a0c689d9daf5aee8283e59b037b78a"} Jul 08 09:08:13.538000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:13.537546 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" event={"ID":"42d8150b-fb0b-4a6e-bae3-0db02436b2a5","Type":"ContainerStarted","Data":"18e2d1fc3695b3eebf1383ec6eda2ec9b85d75e903119dda22f284867799de05"} Jul 08 09:08:14.081409 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.081373 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq"] Jul 08 09:08:14.083489 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.083466 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.086085 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.086058 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 09:08:14.086302 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.086283 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 09:08:14.086412 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.086302 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fdbzt\"" Jul 08 09:08:14.094377 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.094357 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq"] Jul 08 09:08:14.126256 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.126224 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.126404 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.126280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d7c2a543-b027-45f4-b482-15db05f448a3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.227433 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.227400 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.227602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.227459 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d7c2a543-b027-45f4-b482-15db05f448a3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.227602 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.227553 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 09:08:14.227718 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.227623 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert podName:d7c2a543-b027-45f4-b482-15db05f448a3 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:14.727603773 +0000 UTC m=+73.082615315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhdkq" (UID: "d7c2a543-b027-45f4-b482-15db05f448a3") : secret "networking-console-plugin-cert" not found Jul 08 09:08:14.228132 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.228111 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d7c2a543-b027-45f4-b482-15db05f448a3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.541592 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.541549 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" event={"ID":"42d8150b-fb0b-4a6e-bae3-0db02436b2a5","Type":"ContainerStarted","Data":"f927df6c9f1d8d41c31840ddb9f30e7b05c9853e6a4fcf3ff9e7bc8f360d3701"} Jul 08 09:08:14.561021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.560954 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-5svmn" podStartSLOduration=3.294347604 podStartE2EDuration="4.560936789s" totalImagesPullingTime="1.266589185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:14.559897349 +0000 UTC m=+72.914908939" watchObservedRunningTime="2026-07-08 09:08:14.560936789 +0000 UTC m=+72.915948351" Jul 08 09:08:14.631854 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.631818 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:14.632039 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.632007 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:14.632039 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.632029 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86b94ddf7f-xzhhk: secret "image-registry-tls" not found Jul 08 09:08:14.632139 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.632098 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls podName:f785069b-d367-48c5-bfe2-53cc471b0142 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.632077632 +0000 UTC m=+76.987089172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls") pod "image-registry-86b94ddf7f-xzhhk" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142") : secret "image-registry-tls" not found Jul 08 09:08:14.732826 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:14.732784 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:14.733006 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.732970 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 09:08:14.733080 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:14.733043 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert podName:d7c2a543-b027-45f4-b482-15db05f448a3 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:15.733026538 +0000 UTC m=+74.088038084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhdkq" (UID: "d7c2a543-b027-45f4-b482-15db05f448a3") : secret "networking-console-plugin-cert" not found Jul 08 09:08:15.385714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.385671 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:15.547485 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.547446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" event={"ID":"048aa6fd-5839-43dd-a471-1f58a7ab8fd4","Type":"ContainerStarted","Data":"bc64f246d60c6a4b898f4cb067d3a8c87c80346967363af8d174e636face2e1b"} Jul 08 09:08:15.549226 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.549198 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" event={"ID":"9f6e3800-8e80-411b-9506-d6f9f4042f0b","Type":"ContainerStarted","Data":"206e831b01fb9cda04b5080c6bed59f91ded6c342c1bec34ec03ebb3d40443e2"} Jul 08 09:08:15.551663 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.551635 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:15.571276 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.571230 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" podStartSLOduration=2.809423582 podStartE2EDuration="5.571213515s" totalImagesPullingTime="2.761789933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:15.570688712 +0000 UTC m=+73.925700294" watchObservedRunningTime="2026-07-08 09:08:15.571213515 +0000 UTC m=+73.926225073" Jul 08 09:08:15.592611 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.592548 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" podStartSLOduration=2.612793169 podStartE2EDuration="5.592531892s" totalImagesPullingTime="2.979738723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:15.591558313 +0000 UTC m=+73.946569893" watchObservedRunningTime="2026-07-08 09:08:15.592531892 +0000 UTC m=+73.947543458" Jul 08 09:08:15.741890 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:15.741808 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:15.742042 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:15.741947 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 09:08:15.742042 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:15.742011 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert podName:d7c2a543-b027-45f4-b482-15db05f448a3 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:17.741992686 +0000 UTC m=+76.097004230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhdkq" (UID: "d7c2a543-b027-45f4-b482-15db05f448a3") : secret "networking-console-plugin-cert" not found Jul 08 09:08:16.882322 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.882280 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:16.911713 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.911681 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql"] Jul 08 09:08:16.914075 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.914053 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" Jul 08 09:08:16.919314 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.919287 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 09:08:16.919450 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.919314 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nvgl7\"" Jul 08 09:08:16.920379 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.920360 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 09:08:16.931963 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.931941 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql"] Jul 08 09:08:16.955821 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.955798 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8dsm\" (UniqueName: \"kubernetes.io/projected/735b79b5-2139-4483-87b3-368a07110f1e-kube-api-access-x8dsm\") pod \"migrator-d57d58cc9-8kbql\" (UID: \"735b79b5-2139-4483-87b3-368a07110f1e\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" Jul 08 09:08:16.980405 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:16.980371 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.056568 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.056541 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8dsm\" (UniqueName: \"kubernetes.io/projected/735b79b5-2139-4483-87b3-368a07110f1e-kube-api-access-x8dsm\") pod \"migrator-d57d58cc9-8kbql\" (UID: \"735b79b5-2139-4483-87b3-368a07110f1e\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" Jul 08 09:08:17.065019 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.064999 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8dsm\" (UniqueName: \"kubernetes.io/projected/735b79b5-2139-4483-87b3-368a07110f1e-kube-api-access-x8dsm\") pod \"migrator-d57d58cc9-8kbql\" (UID: \"735b79b5-2139-4483-87b3-368a07110f1e\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" Jul 08 09:08:17.081324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.081288 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.222939 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.222850 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" Jul 08 09:08:17.366875 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.366849 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql"] Jul 08 09:08:17.370364 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:17.370332 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735b79b5_2139_4483_87b3_368a07110f1e.slice/crio-7e7e08fadd7ad02fdcc71635d9e9aa6a0938a30c3bc6c84c6bbbe71ed827045b WatchSource:0}: Error finding container 7e7e08fadd7ad02fdcc71635d9e9aa6a0938a30c3bc6c84c6bbbe71ed827045b: Status 404 returned error can't find the container with id 7e7e08fadd7ad02fdcc71635d9e9aa6a0938a30c3bc6c84c6bbbe71ed827045b Jul 08 09:08:17.408510 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.408473 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.527103 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.527029 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.557414 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.557378 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" event={"ID":"735b79b5-2139-4483-87b3-368a07110f1e","Type":"ContainerStarted","Data":"7e7e08fadd7ad02fdcc71635d9e9aa6a0938a30c3bc6c84c6bbbe71ed827045b"} Jul 08 09:08:17.677345 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.677313 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.763776 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.763742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:17.763932 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:17.763891 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 09:08:17.763988 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:17.763969 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert podName:d7c2a543-b027-45f4-b482-15db05f448a3 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:21.763954993 +0000 UTC m=+80.118966533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhdkq" (UID: "d7c2a543-b027-45f4-b482-15db05f448a3") : secret "networking-console-plugin-cert" not found Jul 08 09:08:17.802717 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.802650 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k78ww"] Jul 08 09:08:17.805344 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.805328 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.807348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.807319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.809070 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.809050 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 09:08:17.809174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.809059 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 09:08:17.810176 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.810159 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 09:08:17.810254 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.810174 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 09:08:17.810331 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.810308 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vc9gl\"" Jul 08 09:08:17.837624 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.837598 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k78ww"] Jul 08 09:08:17.864718 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.864691 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.864833 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.864724 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/48daeaa4-f575-471c-8ba1-623ba4be13ff-crio-socket\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.864833 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.864742 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jb8s\" (UniqueName: \"kubernetes.io/projected/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-api-access-8jb8s\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.864833 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.864762 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/48daeaa4-f575-471c-8ba1-623ba4be13ff-data-volume\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.864979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.864881 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.910147 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.910099 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:17.965260 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/48daeaa4-f575-471c-8ba1-623ba4be13ff-crio-socket\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.965392 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jb8s\" (UniqueName: \"kubernetes.io/projected/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-api-access-8jb8s\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.965392 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965281 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/48daeaa4-f575-471c-8ba1-623ba4be13ff-data-volume\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.965392 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965315 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.965392 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965358 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/48daeaa4-f575-471c-8ba1-623ba4be13ff-crio-socket\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.965392 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.965645 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:17.965449 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:17.965645 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:17.965454 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:17.965645 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:17.965516 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.465498022 +0000 UTC m=+76.820509561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : configmap "kube-rbac-proxy" not found Jul 08 09:08:17.965645 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:17.965536 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:18.465526385 +0000 UTC m=+76.820537925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:17.965791 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.965663 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/48daeaa4-f575-471c-8ba1-623ba4be13ff-data-volume\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:17.980178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:17.980153 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jb8s\" (UniqueName: \"kubernetes.io/projected/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-api-access-8jb8s\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:18.469744 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:18.469709 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:18.469977 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:18.469834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:18.469977 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.469885 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:18.470105 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.469975 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:18.470105 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.469986 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:19.469963958 +0000 UTC m=+77.824975518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:18.470105 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.470035 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:19.470018052 +0000 UTC m=+77.825029597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : configmap "kube-rbac-proxy" not found Jul 08 09:08:18.671964 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:18.671930 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:18.672145 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.672078 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:18.672145 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.672095 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86b94ddf7f-xzhhk: secret "image-registry-tls" not found Jul 08 09:08:18.672216 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:18.672152 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls podName:f785069b-d367-48c5-bfe2-53cc471b0142 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:26.672136983 +0000 UTC m=+85.027148523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls") pod "image-registry-86b94ddf7f-xzhhk" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142") : secret "image-registry-tls" not found Jul 08 09:08:19.340236 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.340203 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:19.426254 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.426212 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:19.477809 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.477786 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:19.477958 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.477945 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:19.478000 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:19.477960 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:19.478031 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:19.478006 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:19.478031 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:19.478013 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:21.477994944 +0000 UTC m=+79.833006485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:19.478107 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:19.478040 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:21.47803075 +0000 UTC m=+79.833042289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : configmap "kube-rbac-proxy" not found Jul 08 09:08:19.566114 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.566075 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" event={"ID":"735b79b5-2139-4483-87b3-368a07110f1e","Type":"ContainerStarted","Data":"4ab909c1a76d242e6f23e38c6e2e2ffbf88e592a6625b5ca877a8b83705aeea2"} Jul 08 09:08:19.566278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.566123 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" event={"ID":"735b79b5-2139-4483-87b3-368a07110f1e","Type":"ContainerStarted","Data":"0ae93652ead991b49889feded9b12f98fcfd3b8444d516a4bebd1b936efd7a2c"} Jul 08 09:08:19.702160 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.702128 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:19.805985 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:19.805922 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:20.361044 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:20.361016 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-486wp_e77a0967-6023-4cc7-98da-290dcea1391a/dns-node-resolver/0.log" Jul 08 09:08:20.605761 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:20.605726 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:21.492858 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:21.492827 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:21.493257 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:21.492880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:21.493257 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:21.492971 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:21.493257 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:21.492993 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:21.493257 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:21.493038 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:25.493022496 +0000 UTC m=+83.848034036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : configmap "kube-rbac-proxy" not found Jul 08 09:08:21.493257 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:21.493053 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:25.493047297 +0000 UTC m=+83.848058836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:21.763918 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:21.763830 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-crgh2_ad916d36-3a34-4b04-96dd-d831d52ea3c6/node-ca/0.log" Jul 08 09:08:21.795451 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:21.795425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:21.795618 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:21.795581 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 09:08:21.795662 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:21.795644 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert podName:d7c2a543-b027-45f4-b482-15db05f448a3 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:29.795629143 +0000 UTC m=+88.150640686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhdkq" (UID: "d7c2a543-b027-45f4-b482-15db05f448a3") : secret "networking-console-plugin-cert" not found Jul 08 09:08:22.559518 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:22.559487 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jfrkc_b8c50830-ba2b-427f-ad54-786c4d84b3f7/serve-healthcheck-canary/0.log" Jul 08 09:08:22.759962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:22.759930 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-8kbql_735b79b5-2139-4483-87b3-368a07110f1e/migrator/0.log" Jul 08 09:08:22.959497 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:22.959469 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-8kbql_735b79b5-2139-4483-87b3-368a07110f1e/graceful-termination/0.log" Jul 08 09:08:23.162285 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:23.162171 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-qpjjd_048aa6fd-5839-43dd-a471-1f58a7ab8fd4/kube-storage-version-migrator-operator/0.log" Jul 08 09:08:23.550241 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:23.550209 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:23.957700 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:23.957665 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:24.100989 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:24.100941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:24.294663 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:24.294575 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:24.979809 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:24.979771 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:25.188734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:25.188697 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:25.522380 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:25.522348 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:25.522540 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:25.522507 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 09:08:25.522540 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:25.522519 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:25.522610 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:25.522575 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:33.522556832 +0000 UTC m=+91.877568445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : secret "insights-runtime-extractor-tls" not found Jul 08 09:08:25.526068 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:25.523036 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 09:08:25.526068 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:25.523117 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm podName:48daeaa4-f575-471c-8ba1-623ba4be13ff nodeName:}" failed. No retries permitted until 2026-07-08 09:08:33.523089117 +0000 UTC m=+91.878100736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k78ww" (UID: "48daeaa4-f575-471c-8ba1-623ba4be13ff") : configmap "kube-rbac-proxy" not found Jul 08 09:08:25.798323 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:25.798237 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:25.999154 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:25.999119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:26.089157 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:26.089062 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:26.405877 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:26.405840 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:26.731123 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:26.731040 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:26.731270 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:26.731156 2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 09:08:26.731270 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:26.731168 2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86b94ddf7f-xzhhk: secret "image-registry-tls" not found Jul 08 09:08:26.731270 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:26.731227 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls podName:f785069b-d367-48c5-bfe2-53cc471b0142 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:42.731213151 +0000 UTC m=+101.086224691 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls") pod "image-registry-86b94ddf7f-xzhhk" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142") : secret "image-registry-tls" not found Jul 08 09:08:27.482055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:27.482017 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:27.583682 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:27.583654 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:27.699153 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:27.699121 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:28.054516 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:28.054484 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:28.363690 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:28.363654 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:28.632746 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:28.632667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:28.795087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:28.795052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:28.889300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:28.889210 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:29.238947 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:29.238843 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:29.792526 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:29.792491 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:29.854245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:29.854216 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:29.854382 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:29.854325 2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 09:08:29.854382 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:29.854378 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert podName:d7c2a543-b027-45f4-b482-15db05f448a3 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:45.854364159 +0000 UTC m=+104.209375698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhdkq" (UID: "d7c2a543-b027-45f4-b482-15db05f448a3") : secret "networking-console-plugin-cert" not found Jul 08 09:08:30.155590 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:30.155557 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:30.397601 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:30.397569 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:30.491098 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:30.491014 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:30.606618 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:30.606571 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:30.796245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:30.796161 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:31.187823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:31.187790 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:31.778676 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:31.778640 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:31.969292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:31.969258 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:32.179770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:32.179740 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:32.574943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:32.574839 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:32.780816 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:32.780783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:33.181178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.181148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:33.582955 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.582841 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:33.582955 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.582929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:33.583444 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.583417 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/48daeaa4-f575-471c-8ba1-623ba4be13ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:33.585229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.585207 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/48daeaa4-f575-471c-8ba1-623ba4be13ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k78ww\" (UID: \"48daeaa4-f575-471c-8ba1-623ba4be13ff\") " pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:33.714644 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.714603 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k78ww" Jul 08 09:08:33.787346 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.787301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:33.843245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.843150 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8kbql" podStartSLOduration=15.885173781 podStartE2EDuration="17.843128564s" totalImagesPullingTime="1.957954783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:19.584540787 +0000 UTC m=+77.939552349" watchObservedRunningTime="2026-07-08 09:08:33.843128564 +0000 UTC m=+92.198140127" Jul 08 09:08:33.843680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.843662 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k78ww"] Jul 08 09:08:33.846569 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:33.846546 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48daeaa4_f575_471c_8ba1_623ba4be13ff.slice/crio-acd48d3399a6ae5e13640a0c3d7f8b3d08db6541a822e487f627fbab1ae648e6 WatchSource:0}: Error finding container acd48d3399a6ae5e13640a0c3d7f8b3d08db6541a822e487f627fbab1ae648e6: Status 404 returned error can't find the container with id acd48d3399a6ae5e13640a0c3d7f8b3d08db6541a822e487f627fbab1ae648e6 Jul 08 09:08:33.984014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:33.983975 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:34.196121 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:34.196091 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:34.617784 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:34.617748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k78ww" event={"ID":"48daeaa4-f575-471c-8ba1-623ba4be13ff","Type":"ContainerStarted","Data":"acd48d3399a6ae5e13640a0c3d7f8b3d08db6541a822e487f627fbab1ae648e6"} Jul 08 09:08:37.629764 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:37.629724 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k78ww" event={"ID":"48daeaa4-f575-471c-8ba1-623ba4be13ff","Type":"ContainerStarted","Data":"d62aa33746fc8f885f0bac278538409caf6085420f9b2f107ae8dce1d169d1f9"} Jul 08 09:08:38.634730 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:38.634691 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k78ww" event={"ID":"48daeaa4-f575-471c-8ba1-623ba4be13ff","Type":"ContainerStarted","Data":"d59fde792fde099d2d0fd4fb4ca6d11b3927bf38a06922f41f0e5ce271b7f86e"} Jul 08 09:08:40.342662 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.342619 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:08:40.345278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.345251 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa74e311-430b-42f8-bc39-dd43f477309c-metrics-tls\") pod \"dns-default-zf8b5\" (UID: \"fa74e311-430b-42f8-bc39-dd43f477309c\") " pod="openshift-dns/dns-default-zf8b5" Jul 08 09:08:40.370881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.370860 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qglp2\"" Jul 08 09:08:40.378419 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.378402 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zf8b5" Jul 08 09:08:40.528922 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.528828 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zf8b5"] Jul 08 09:08:40.533123 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:40.533094 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa74e311_430b_42f8_bc39_dd43f477309c.slice/crio-d621bedb01a7d63d77ee7947e2d38b216f7d7f5d7066edaddb19c9f1bd4ad2e0 WatchSource:0}: Error finding container d621bedb01a7d63d77ee7947e2d38b216f7d7f5d7066edaddb19c9f1bd4ad2e0: Status 404 returned error can't find the container with id d621bedb01a7d63d77ee7947e2d38b216f7d7f5d7066edaddb19c9f1bd4ad2e0 Jul 08 09:08:40.641732 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.641696 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k78ww" event={"ID":"48daeaa4-f575-471c-8ba1-623ba4be13ff","Type":"ContainerStarted","Data":"7f8587c1ffa17a898ed00cf4396c89759bd7049a015972fb8e0950190c903061"} Jul 08 09:08:40.642735 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.642712 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zf8b5" event={"ID":"fa74e311-430b-42f8-bc39-dd43f477309c","Type":"ContainerStarted","Data":"d621bedb01a7d63d77ee7947e2d38b216f7d7f5d7066edaddb19c9f1bd4ad2e0"} Jul 08 09:08:40.668140 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:40.668098 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k78ww" podStartSLOduration=17.165234613 podStartE2EDuration="23.668083455s" totalImagesPullingTime="6.502848842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:08:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:40.665956014 +0000 UTC m=+99.020967594" watchObservedRunningTime="2026-07-08 09:08:40.668083455 +0000 UTC m=+99.023095017" Jul 08 09:08:42.760350 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:42.760304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:42.763049 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:42.763022 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"image-registry-86b94ddf7f-xzhhk\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:42.776951 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:42.776920 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:42.921263 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:42.921227 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86b94ddf7f-xzhhk"] Jul 08 09:08:43.370134 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:43.370098 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf785069b_d367_48c5_bfe2_53cc471b0142.slice/crio-de66efbc6954bb7c75b0da323cb5e156a333fec9a7ca68687e07202235444717 WatchSource:0}: Error finding container de66efbc6954bb7c75b0da323cb5e156a333fec9a7ca68687e07202235444717: Status 404 returned error can't find the container with id de66efbc6954bb7c75b0da323cb5e156a333fec9a7ca68687e07202235444717 Jul 08 09:08:43.459281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.459256 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86b94ddf7f-xzhhk"] Jul 08 09:08:43.488036 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.488009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:43.522361 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.522333 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86f45cdf87-khsjs"] Jul 08 09:08:43.531135 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.531114 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.540209 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.540193 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-84spb" Jul 08 09:08:43.544440 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.544420 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f45cdf87-khsjs"] Jul 08 09:08:43.583398 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.583351 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:43.655413 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.655376 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" event={"ID":"f785069b-d367-48c5-bfe2-53cc471b0142","Type":"ContainerStarted","Data":"6d4ffb10acaae3b0d887e55e186f0fa71df03f50da200cf7d61a594295e0450f"} Jul 08 09:08:43.655575 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.655427 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" event={"ID":"f785069b-d367-48c5-bfe2-53cc471b0142","Type":"ContainerStarted","Data":"de66efbc6954bb7c75b0da323cb5e156a333fec9a7ca68687e07202235444717"} Jul 08 09:08:43.655575 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.655466 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:08:43.669339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9843a1ff-3861-4e32-ba99-9881307906f2-ca-trust-extracted\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669169 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz8qr\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-kube-api-access-vz8qr\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669203 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-bound-sa-token\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669251 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9843a1ff-3861-4e32-ba99-9881307906f2-image-registry-private-configuration\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669329 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9843a1ff-3861-4e32-ba99-9881307906f2-installation-pull-secrets\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669696 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669358 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9843a1ff-3861-4e32-ba99-9881307906f2-registry-certificates\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669696 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669405 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-registry-tls\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.669696 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.669433 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9843a1ff-3861-4e32-ba99-9881307906f2-trusted-ca\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.688493 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.688436 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" podStartSLOduration=33.688416552 podStartE2EDuration="33.688416552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:43.687520309 +0000 UTC m=+102.042531871" watchObservedRunningTime="2026-07-08 09:08:43.688416552 +0000 UTC m=+102.043428116" Jul 08 09:08:43.771026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.770749 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz8qr\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-kube-api-access-vz8qr\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.771026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.770795 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-bound-sa-token\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.771026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.770835 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9843a1ff-3861-4e32-ba99-9881307906f2-image-registry-private-configuration\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.771026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.770894 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9843a1ff-3861-4e32-ba99-9881307906f2-installation-pull-secrets\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.771026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.770944 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9843a1ff-3861-4e32-ba99-9881307906f2-registry-certificates\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.771026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.770980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-registry-tls\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.772980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.771746 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9843a1ff-3861-4e32-ba99-9881307906f2-trusted-ca\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.772980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.771819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9843a1ff-3861-4e32-ba99-9881307906f2-ca-trust-extracted\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.772980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.772284 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9843a1ff-3861-4e32-ba99-9881307906f2-registry-certificates\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.772980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.772303 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9843a1ff-3861-4e32-ba99-9881307906f2-ca-trust-extracted\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.773311 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.773263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9843a1ff-3861-4e32-ba99-9881307906f2-trusted-ca\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.775048 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.775024 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9843a1ff-3861-4e32-ba99-9881307906f2-image-registry-private-configuration\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.775674 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.775649 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-registry-tls\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.776112 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.776088 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9843a1ff-3861-4e32-ba99-9881307906f2-installation-pull-secrets\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.782210 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.782171 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-bound-sa-token\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.782210 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.782196 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz8qr\" (UniqueName: \"kubernetes.io/projected/9843a1ff-3861-4e32-ba99-9881307906f2-kube-api-access-vz8qr\") pod \"image-registry-86f45cdf87-khsjs\" (UID: \"9843a1ff-3861-4e32-ba99-9881307906f2\") " pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.839422 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.839315 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:43.999161 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:43.999130 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f45cdf87-khsjs"] Jul 08 09:08:44.002085 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:44.002061 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9843a1ff_3861_4e32_ba99_9881307906f2.slice/crio-35e8a93791ba475b09f0ea60259c6f96ac08a2b93d5967ac424214604566d7b1 WatchSource:0}: Error finding container 35e8a93791ba475b09f0ea60259c6f96ac08a2b93d5967ac424214604566d7b1: Status 404 returned error can't find the container with id 35e8a93791ba475b09f0ea60259c6f96ac08a2b93d5967ac424214604566d7b1 Jul 08 09:08:44.659562 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.659527 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" event={"ID":"9843a1ff-3861-4e32-ba99-9881307906f2","Type":"ContainerStarted","Data":"549972235fed2be4d7a09bca244e1e694453300c228996fc456373a22a1cfea8"} Jul 08 09:08:44.659768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.659581 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" event={"ID":"9843a1ff-3861-4e32-ba99-9881307906f2","Type":"ContainerStarted","Data":"35e8a93791ba475b09f0ea60259c6f96ac08a2b93d5967ac424214604566d7b1"} Jul 08 09:08:44.659768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.659636 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:08:44.661298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.661276 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zf8b5" event={"ID":"fa74e311-430b-42f8-bc39-dd43f477309c","Type":"ContainerStarted","Data":"f60ce224d4ef1a7cc3642016db3a3a0a51675f0430deaca72401659c71bc5963"} Jul 08 09:08:44.661410 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.661318 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zf8b5" event={"ID":"fa74e311-430b-42f8-bc39-dd43f477309c","Type":"ContainerStarted","Data":"62f0a89bb7c8ba515b7d4a35bd25911850528c75e12519e7d6253fed3b31e613"} Jul 08 09:08:44.661410 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.661401 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zf8b5" Jul 08 09:08:44.714505 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.714459 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zf8b5" podStartSLOduration=65.632416628 podStartE2EDuration="1m8.714441552s" totalImagesPullingTime="3.082024924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:44.714089312 +0000 UTC m=+103.069100868" watchObservedRunningTime="2026-07-08 09:08:44.714441552 +0000 UTC m=+103.069453114" Jul 08 09:08:44.714702 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:44.714625 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" podStartSLOduration=1.7146191769999999 podStartE2EDuration="1.714619177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:44.686335272 +0000 UTC m=+103.041346833" watchObservedRunningTime="2026-07-08 09:08:44.714619177 +0000 UTC m=+103.069630740" Jul 08 09:08:45.891628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:45.891593 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:45.894028 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:45.894004 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d7c2a543-b027-45f4-b482-15db05f448a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhdkq\" (UID: \"d7c2a543-b027-45f4-b482-15db05f448a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:46.099950 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.099915 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:46.193912 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.193832 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" Jul 08 09:08:46.241275 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.241245 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:46.334000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.333940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:46.342354 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.342332 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq"] Jul 08 09:08:46.345767 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:46.345741 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c2a543_b027_45f4_b482_15db05f448a3.slice/crio-4a9c97a2739ef9b7226ee007e380ec4c9eb2ef08ac6896ad4002ee5690f1c734 WatchSource:0}: Error finding container 4a9c97a2739ef9b7226ee007e380ec4c9eb2ef08ac6896ad4002ee5690f1c734: Status 404 returned error can't find the container with id 4a9c97a2739ef9b7226ee007e380ec4c9eb2ef08ac6896ad4002ee5690f1c734 Jul 08 09:08:46.437615 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.437578 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:46.672641 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:46.672609 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" event={"ID":"d7c2a543-b027-45f4-b482-15db05f448a3","Type":"ContainerStarted","Data":"4a9c97a2739ef9b7226ee007e380ec4c9eb2ef08ac6896ad4002ee5690f1c734"} Jul 08 09:08:48.681716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:48.681680 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" event={"ID":"d7c2a543-b027-45f4-b482-15db05f448a3","Type":"ContainerStarted","Data":"5dbea57df426db7344368aa346acdd49667bd9018fcdd659076402f3067413c1"} Jul 08 09:08:48.705895 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:48.705849 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhdkq" podStartSLOduration=32.76037693 podStartE2EDuration="34.705835179s" totalImagesPullingTime="1.945458249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:48.705718077 +0000 UTC m=+107.060729665" watchObservedRunningTime="2026-07-08 09:08:48.705835179 +0000 UTC m=+107.060846742" Jul 08 09:08:48.960317 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:48.960223 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:49.039887 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:49.039856 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:50.328313 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.328280 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:50.394111 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.394082 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt"] Jul 08 09:08:50.397138 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.397122 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.400774 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.400745 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-97lzs\"" Jul 08 09:08:50.400917 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.400780 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 09:08:50.400917 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.400790 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 09:08:50.400917 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.400754 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 09:08:50.401091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.400922 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 09:08:50.401091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.400924 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 09:08:50.415330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.415302 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt"] Jul 08 09:08:50.430874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.430851 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:50.524742 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.524715 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.524868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.524750 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.524868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.524767 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.524981 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.524887 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt6qg\" (UniqueName: \"kubernetes.io/projected/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-kube-api-access-jt6qg\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.626174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.626149 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.626330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.626187 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.626330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.626204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.626330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.626238 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt6qg\" (UniqueName: \"kubernetes.io/projected/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-kube-api-access-jt6qg\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.626813 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.626793 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.628569 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.628542 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.628649 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.628628 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.638808 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.638787 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt6qg\" (UniqueName: \"kubernetes.io/projected/0f69d47e-67ab-4cd8-b2e2-d9ce835c8910-kube-api-access-jt6qg\") pod \"prometheus-operator-567f7b8b7c-9s2wt\" (UID: \"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.677515 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.677494 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:50.706020 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.706001 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" Jul 08 09:08:50.842232 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:50.842202 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt"] Jul 08 09:08:50.845849 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:50.845825 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f69d47e_67ab_4cd8_b2e2_d9ce835c8910.slice/crio-dc0b41809ca4c25a844961e8cc77b7bf379f3522c02b7684b0fe5a25421a4abb WatchSource:0}: Error finding container dc0b41809ca4c25a844961e8cc77b7bf379f3522c02b7684b0fe5a25421a4abb: Status 404 returned error can't find the container with id dc0b41809ca4c25a844961e8cc77b7bf379f3522c02b7684b0fe5a25421a4abb Jul 08 09:08:51.695502 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:51.695460 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" event={"ID":"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910","Type":"ContainerStarted","Data":"dc0b41809ca4c25a844961e8cc77b7bf379f3522c02b7684b0fe5a25421a4abb"} Jul 08 09:08:52.510407 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:52.510318 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:52.700724 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:52.700682 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" event={"ID":"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910","Type":"ContainerStarted","Data":"6733c3df06092d57c58715252db99d2fd88b2c66ebb8a4f45b50b4182144ef47"} Jul 08 09:08:52.701108 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:52.700753 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" event={"ID":"0f69d47e-67ab-4cd8-b2e2-d9ce835c8910","Type":"ContainerStarted","Data":"e4d06a667cb254231d683ef85914e9b9c09c4a1ac4330d3da4bf05ed8c9d2153"} Jul 08 09:08:52.719398 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:52.719356 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9s2wt" podStartSLOduration=1.3659105089999999 podStartE2EDuration="2.719341826s" totalImagesPullingTime="1.353431317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:52.717932699 +0000 UTC m=+111.072944261" watchObservedRunningTime="2026-07-08 09:08:52.719341826 +0000 UTC m=+111.074353388" Jul 08 09:08:52.940126 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:52.940095 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:53.004597 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:53.004570 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:53.886396 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:53.886364 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:53.959091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:53.959051 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.046935 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.046885 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.373152 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.373100 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.557372 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.557341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.670596 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.670523 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zf8b5" Jul 08 09:08:54.680890 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.680866 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.787756 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.787719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.812455 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.812429 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v"] Jul 08 09:08:54.814719 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.814706 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:54.818664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.818642 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 09:08:54.818832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.818815 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dwmqv\"" Jul 08 09:08:54.819202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.819186 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 09:08:54.819537 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.819522 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 09:08:54.832222 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.832198 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v"] Jul 08 09:08:54.877772 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.877745 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7"] Jul 08 09:08:54.879791 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.879773 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:54.884375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.884356 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 09:08:54.884977 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.884961 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 09:08:54.885629 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.885614 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9dksb\"" Jul 08 09:08:54.902447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.902428 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7"] Jul 08 09:08:54.907366 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.907344 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:54.939088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.939024 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k2q9v"] Jul 08 09:08:54.941272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.941255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:54.943938 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.943913 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 09:08:54.944035 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.944022 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 09:08:54.944375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.944362 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tzxvj\"" Jul 08 09:08:54.945362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.945349 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 09:08:54.961856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.961830 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:54.961959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.961869 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxqjk\" (UniqueName: \"kubernetes.io/projected/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-api-access-jxqjk\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:54.961959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.961891 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:54.961959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.961944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:54.962071 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.961971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62cf7623-90e6-4c31-8c79-5d5349af1c1f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:54.962071 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:54.961987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62cf7623-90e6-4c31-8c79-5d5349af1c1f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.011446 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.011423 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:55.063346 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063325 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.063460 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063353 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-wtmp\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.063460 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063376 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.063460 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.063564 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-tls\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.063564 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-textfile\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.063564 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0de23a3-a3fc-4d89-80f4-415886dd6e02-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.063652 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063591 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.063652 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063616 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8787e42a-b40c-46e5-9a30-b13010103b67-metrics-client-ca\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.063714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063652 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxqjk\" (UniqueName: \"kubernetes.io/projected/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-api-access-jxqjk\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.063714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063670 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.063714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063698 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-sys\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.063837 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063814 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.063934 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063856 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvfvm\" (UniqueName: \"kubernetes.io/projected/b0de23a3-a3fc-4d89-80f4-415886dd6e02-kube-api-access-jvfvm\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.064050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.063978 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-accelerators-collector-config\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.064050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064013 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb8tc\" (UniqueName: \"kubernetes.io/projected/8787e42a-b40c-46e5-9a30-b13010103b67-kube-api-access-mb8tc\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.064050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064046 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62cf7623-90e6-4c31-8c79-5d5349af1c1f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.064196 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064062 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-root\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.064196 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62cf7623-90e6-4c31-8c79-5d5349af1c1f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.064403 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064385 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62cf7623-90e6-4c31-8c79-5d5349af1c1f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.064482 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064455 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.064832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.064812 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62cf7623-90e6-4c31-8c79-5d5349af1c1f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.066169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.066151 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.066169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.066165 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.081542 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.081518 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxqjk\" (UniqueName: \"kubernetes.io/projected/62cf7623-90e6-4c31-8c79-5d5349af1c1f-kube-api-access-jxqjk\") pod \"kube-state-metrics-7c9dbb8b7d-8qh2v\" (UID: \"62cf7623-90e6-4c31-8c79-5d5349af1c1f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.122646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.122619 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:55.123486 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.123464 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" Jul 08 09:08:55.165555 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-sys\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165561 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvfvm\" (UniqueName: \"kubernetes.io/projected/b0de23a3-a3fc-4d89-80f4-415886dd6e02-kube-api-access-jvfvm\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.165710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165583 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-accelerators-collector-config\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165644 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb8tc\" (UniqueName: \"kubernetes.io/projected/8787e42a-b40c-46e5-9a30-b13010103b67-kube-api-access-mb8tc\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165697 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-root\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165724 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-sys\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165735 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.165929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-wtmp\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165791 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-root\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.165929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165804 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.165929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165947 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-tls\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.165952 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-wtmp\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:55.165873 2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-textfile\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:08:55.166032 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-tls podName:b0de23a3-a3fc-4d89-80f4-415886dd6e02 nodeName:}" failed. No retries permitted until 2026-07-08 09:08:55.666012764 +0000 UTC m=+114.021024306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-nn5d7" (UID: "b0de23a3-a3fc-4d89-80f4-415886dd6e02") : secret "openshift-state-metrics-tls" not found Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0de23a3-a3fc-4d89-80f4-415886dd6e02-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.166221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8787e42a-b40c-46e5-9a30-b13010103b67-metrics-client-ca\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166239 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-accelerators-collector-config\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166272 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-textfile\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166682 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8787e42a-b40c-46e5-9a30-b13010103b67-metrics-client-ca\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.166938 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.166916 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0de23a3-a3fc-4d89-80f4-415886dd6e02-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.168587 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.168551 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.168587 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.168581 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.169287 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.169264 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8787e42a-b40c-46e5-9a30-b13010103b67-node-exporter-tls\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.179600 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.179575 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb8tc\" (UniqueName: \"kubernetes.io/projected/8787e42a-b40c-46e5-9a30-b13010103b67-kube-api-access-mb8tc\") pod \"node-exporter-k2q9v\" (UID: \"8787e42a-b40c-46e5-9a30-b13010103b67\") " pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.182261 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.182241 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvfvm\" (UniqueName: \"kubernetes.io/projected/b0de23a3-a3fc-4d89-80f4-415886dd6e02-kube-api-access-jvfvm\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.231964 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.231934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:55.250267 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.250240 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k2q9v" Jul 08 09:08:55.256707 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.256682 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v"] Jul 08 09:08:55.259981 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:55.259952 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cf7623_90e6_4c31_8c79_5d5349af1c1f.slice/crio-49f3c36e866ed8329d2b56aa34e693072677b9f5bd93332d144ee7a017a803ae WatchSource:0}: Error finding container 49f3c36e866ed8329d2b56aa34e693072677b9f5bd93332d144ee7a017a803ae: Status 404 returned error can't find the container with id 49f3c36e866ed8329d2b56aa34e693072677b9f5bd93332d144ee7a017a803ae Jul 08 09:08:55.260784 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:55.260757 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8787e42a_b40c_46e5_9a30_b13010103b67.slice/crio-2322d918b9a3bda57f22f0d40cf42eebc815569a19e6aa810e2eba5d15c8e23e WatchSource:0}: Error finding container 2322d918b9a3bda57f22f0d40cf42eebc815569a19e6aa810e2eba5d15c8e23e: Status 404 returned error can't find the container with id 2322d918b9a3bda57f22f0d40cf42eebc815569a19e6aa810e2eba5d15c8e23e Jul 08 09:08:55.355300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.355271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:55.669882 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.669849 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.672168 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.672150 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b0de23a3-a3fc-4d89-80f4-415886dd6e02-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nn5d7\" (UID: \"b0de23a3-a3fc-4d89-80f4-415886dd6e02\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.716199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.716150 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" event={"ID":"62cf7623-90e6-4c31-8c79-5d5349af1c1f","Type":"ContainerStarted","Data":"49f3c36e866ed8329d2b56aa34e693072677b9f5bd93332d144ee7a017a803ae"} Jul 08 09:08:55.718439 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.717392 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k2q9v" event={"ID":"8787e42a-b40c-46e5-9a30-b13010103b67","Type":"ContainerStarted","Data":"2322d918b9a3bda57f22f0d40cf42eebc815569a19e6aa810e2eba5d15c8e23e"} Jul 08 09:08:55.788605 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.788578 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" Jul 08 09:08:55.834184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.834147 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:55.930542 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.930475 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:08:55.940554 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:55.939991 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0de23a3_a3fc_4d89_80f4_415886dd6e02.slice/crio-216c7763c03ab4986e961011fb41018a913afd735377740f23e7216c433b7516 WatchSource:0}: Error finding container 216c7763c03ab4986e961011fb41018a913afd735377740f23e7216c433b7516: Status 404 returned error can't find the container with id 216c7763c03ab4986e961011fb41018a913afd735377740f23e7216c433b7516 Jul 08 09:08:55.940934 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.940830 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7"] Jul 08 09:08:55.941088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.940963 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:55.942170 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.942101 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:55.944162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.943691 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 09:08:55.944162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.943856 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 09:08:55.944162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.943884 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t7pbl\"" Jul 08 09:08:55.944162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.943991 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 09:08:55.944162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.944071 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 09:08:55.944454 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.944319 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 09:08:55.944454 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.944404 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 09:08:55.944551 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.944476 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 09:08:55.944609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.944550 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 09:08:55.946531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.946517 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 09:08:55.957356 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:55.957318 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:08:56.073375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073309 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073575 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073402 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073575 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073500 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2xxd\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-kube-api-access-j2xxd\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073688 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073596 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073688 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073634 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-out\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073688 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073677 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073704 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073773 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073802 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.073832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073828 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-volume\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.074059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073857 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-web-config\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.074059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.073894 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174462 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xxd\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-kube-api-access-j2xxd\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174626 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174486 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174697 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174638 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-out\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174749 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174696 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174749 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174872 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174761 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174872 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174804 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174872 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-volume\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.174872 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174860 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-web-config\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.175087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174891 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.175087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174936 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.175087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.174966 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.175087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.175016 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.175546 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.175295 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.175959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.175934 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.176143 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.176120 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.179191 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.178454 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-out\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.179191 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.178551 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.179191 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.178715 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.179191 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.179088 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.179191 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.179160 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.179478 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.179432 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-volume\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.180004 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.179974 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.180366 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.180331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-web-config\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.181353 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.181289 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.183666 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.183623 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2xxd\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-kube-api-access-j2xxd\") pod \"alertmanager-main-0\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.255198 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.255161 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:08:56.596644 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.596561 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:56.713566 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.713530 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:56.724055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.724025 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" event={"ID":"b0de23a3-a3fc-4d89-80f4-415886dd6e02","Type":"ContainerStarted","Data":"cdce33c0508270981bda494c189489e5e2b033ad3130de59bb7d778ec66c5e84"} Jul 08 09:08:56.724207 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.724093 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" event={"ID":"b0de23a3-a3fc-4d89-80f4-415886dd6e02","Type":"ContainerStarted","Data":"12b57ed4720b2b40eef10ce22a2d405a44417ab2fb1cbb57bfeb9a60216caad6"} Jul 08 09:08:56.724207 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.724125 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" event={"ID":"b0de23a3-a3fc-4d89-80f4-415886dd6e02","Type":"ContainerStarted","Data":"216c7763c03ab4986e961011fb41018a913afd735377740f23e7216c433b7516"} Jul 08 09:08:56.996917 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:56.996785 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:08:57.011230 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:57.011202 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470eb0c6_3326_478b_8d6d_008f39dd0ec9.slice/crio-64594a6583267145e09456a49a228130ba6fac97c81b555674fe02e4d153ae81 WatchSource:0}: Error finding container 64594a6583267145e09456a49a228130ba6fac97c81b555674fe02e4d153ae81: Status 404 returned error can't find the container with id 64594a6583267145e09456a49a228130ba6fac97c81b555674fe02e4d153ae81 Jul 08 09:08:57.730274 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.730190 2572 generic.go:356] "Generic (PLEG): container finished" podID="8787e42a-b40c-46e5-9a30-b13010103b67" containerID="992084537f978cfd2d071cc78630b2b5c52355e9315c37e7008fc843b4edd2a4" exitCode=0 Jul 08 09:08:57.730430 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.730271 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k2q9v" event={"ID":"8787e42a-b40c-46e5-9a30-b13010103b67","Type":"ContainerDied","Data":"992084537f978cfd2d071cc78630b2b5c52355e9315c37e7008fc843b4edd2a4"} Jul 08 09:08:57.733234 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.733062 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" event={"ID":"b0de23a3-a3fc-4d89-80f4-415886dd6e02","Type":"ContainerStarted","Data":"409d255933db1629ab6f2faa56f2ce96e20bfbae7d6a10eef2f59842dfc6e6e3"} Jul 08 09:08:57.735710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.735674 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" event={"ID":"62cf7623-90e6-4c31-8c79-5d5349af1c1f","Type":"ContainerStarted","Data":"1515477915e3f0b063a7765e76e24e559b3720d528f67917133ba2fae3aec770"} Jul 08 09:08:57.735847 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.735752 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" event={"ID":"62cf7623-90e6-4c31-8c79-5d5349af1c1f","Type":"ContainerStarted","Data":"0355f74408e81eac167a658134f583561b9db12e85f1521fabfcd73fa349c554"} Jul 08 09:08:57.735847 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.735774 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" event={"ID":"62cf7623-90e6-4c31-8c79-5d5349af1c1f","Type":"ContainerStarted","Data":"e45107814a1a0e948702142204ec4e191ae5613c8ffe952ae5c7120f50f7fb9d"} Jul 08 09:08:57.737294 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.737273 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"64594a6583267145e09456a49a228130ba6fac97c81b555674fe02e4d153ae81"} Jul 08 09:08:57.776617 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.774299 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-8qh2v" podStartSLOduration=2.249276719 podStartE2EDuration="3.774278562s" totalImagesPullingTime="1.525001843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:57.771406999 +0000 UTC m=+116.126418557" watchObservedRunningTime="2026-07-08 09:08:57.774278562 +0000 UTC m=+116.129290127" Jul 08 09:08:57.796827 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.796771 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nn5d7" podStartSLOduration=2.452473967 podStartE2EDuration="3.796749495s" totalImagesPullingTime="1.344275528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:57.795330031 +0000 UTC m=+116.150341596" watchObservedRunningTime="2026-07-08 09:08:57.796749495 +0000 UTC m=+116.151761058" Jul 08 09:08:57.892114 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.892088 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:57.972322 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:57.972243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:58.601731 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.601643 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:58.723005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.722978 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:58.741761 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.741729 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4" exitCode=0 Jul 08 09:08:58.741943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.741807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4"} Jul 08 09:08:58.744026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.744001 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k2q9v" event={"ID":"8787e42a-b40c-46e5-9a30-b13010103b67","Type":"ContainerStarted","Data":"630f352018b089aabb2bf59655c167eaa10b5a01b11aa0dbd243267703d51f27"} Jul 08 09:08:58.744119 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.744053 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k2q9v" event={"ID":"8787e42a-b40c-46e5-9a30-b13010103b67","Type":"ContainerStarted","Data":"3a4e9f4f9a0131acffe3a1a12293e706d2c4bfe5a2de1af95a002cb09eb67430"} Jul 08 09:08:58.816852 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:58.816798 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k2q9v" podStartSLOduration=3.216423649 podStartE2EDuration="4.816778694s" totalImagesPullingTime="1.522951811s" totalInitContainerRuntime="77.403234ms" isStatefulPod=true podCreationTimestamp="2026-07-08 09:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:08:58.816533615 +0000 UTC m=+117.171545177" watchObservedRunningTime="2026-07-08 09:08:58.816778694 +0000 UTC m=+117.171790261" Jul 08 09:08:59.301953 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.301921 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8699564ddd-krr97"] Jul 08 09:08:59.304996 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.304974 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.307845 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.307825 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 09:08:59.307969 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.307828 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 09:08:59.308961 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.308897 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-38dgubiru5ecr\"" Jul 08 09:08:59.308961 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.308933 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fvqzj\"" Jul 08 09:08:59.308961 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.308943 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 09:08:59.309190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.308946 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 09:08:59.317415 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.317397 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8699564ddd-krr97"] Jul 08 09:08:59.405255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405217 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-secret-metrics-server-client-certs\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.405447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405296 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-metrics-server-audit-profiles\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.405447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405328 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-audit-log\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.405447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78xs8\" (UniqueName: \"kubernetes.io/projected/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-kube-api-access-78xs8\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.405447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405426 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-secret-metrics-server-tls\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.405673 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.405673 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.405559 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-client-ca-bundle\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506684 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506657 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506684 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506687 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-client-ca-bundle\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-secret-metrics-server-client-certs\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506767 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-metrics-server-audit-profiles\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506789 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-audit-log\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506803 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78xs8\" (UniqueName: \"kubernetes.io/projected/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-kube-api-access-78xs8\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.506919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.506823 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-secret-metrics-server-tls\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.507465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.507404 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-audit-log\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.507772 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.507750 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.507828 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.507772 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-metrics-server-audit-profiles\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.509286 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.509263 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-client-ca-bundle\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.509656 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.509629 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-secret-metrics-server-client-certs\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.509798 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.509778 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-secret-metrics-server-tls\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.511608 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.511581 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:59.516518 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.516496 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78xs8\" (UniqueName: \"kubernetes.io/projected/efd395c9-49c7-4b95-a7e6-c8d40d5cb19c-kube-api-access-78xs8\") pod \"metrics-server-8699564ddd-krr97\" (UID: \"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c\") " pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.586921 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.586834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:08:59.614071 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.614044 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:08:59.741472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.741307 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8699564ddd-krr97"] Jul 08 09:08:59.743933 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:08:59.743894 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd395c9_49c7_4b95_a7e6_c8d40d5cb19c.slice/crio-22488206517bf27581dca8b79c541460906bedd0c0df3fa09c40b8e3cc93c588 WatchSource:0}: Error finding container 22488206517bf27581dca8b79c541460906bedd0c0df3fa09c40b8e3cc93c588: Status 404 returned error can't find the container with id 22488206517bf27581dca8b79c541460906bedd0c0df3fa09c40b8e3cc93c588 Jul 08 09:08:59.749318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:08:59.749262 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" event={"ID":"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c","Type":"ContainerStarted","Data":"22488206517bf27581dca8b79c541460906bedd0c0df3fa09c40b8e3cc93c588"} Jul 08 09:09:00.458333 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:00.458306 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:00.757629 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:00.757597 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538"} Jul 08 09:09:00.758032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:00.757670 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787"} Jul 08 09:09:00.758032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:00.757701 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7"} Jul 08 09:09:00.758032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:00.757726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443"} Jul 08 09:09:00.758032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:00.757748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8"} Jul 08 09:09:01.211783 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.211735 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:01.220079 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.220055 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:09:01.225063 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.225038 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.227472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.227450 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232536 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232649 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232670 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232759 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232794 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232804 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 09:09:01.232919 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.232669 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 09:09:01.233332 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.233311 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 09:09:01.234041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.233938 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 09:09:01.234041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.234032 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 09:09:01.234263 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.234241 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 09:09:01.235531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.235027 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xbl7k\"" Jul 08 09:09:01.235531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.235219 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fi8bge120ki7h\"" Jul 08 09:09:01.235531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.235406 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 09:09:01.237385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.237370 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 09:09:01.257945 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.257896 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329217 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329293 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329330 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329410 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329434 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw729\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-kube-api-access-kw729\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329458 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329492 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-config-out\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.329595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.329526 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.330848 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.330445 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.330848 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.330488 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-config\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.330525 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-web-config\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331708 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331736 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331763 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331788 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.331987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.331821 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433099 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433069 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433111 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433135 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw729\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-kube-api-access-kw729\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433191 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-config-out\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433239 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433271 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-config\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433297 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433326 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433349 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-web-config\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433408 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433432 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433456 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433558 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.433599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.434124 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.433643 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.434183 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.434122 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.434252 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.434230 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.434313 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.434230 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.437465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.437283 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.437607 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.437517 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-config-out\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.437607 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.437586 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441376 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.438117 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441376 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.438537 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-config\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441376 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.438959 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441376 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.441207 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441591 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.441500 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441641 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.441609 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.441864 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.441836 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.442394 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.442371 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.442532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.442451 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-web-config\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.443307 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.443228 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.443844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.443818 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.444220 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.444051 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.444861 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.444780 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.446594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.446571 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw729\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-kube-api-access-kw729\") pod \"prometheus-k8s-0\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.540395 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.540315 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:01.967374 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:01.967334 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:09:01.970494 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:09:01.970469 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb9c7e1_edb4_4e36_88ab_db418428e340.slice/crio-5aa2dd1af2c148b0535e7285217f97493fc1bf9241fbad915147528e96e0204a WatchSource:0}: Error finding container 5aa2dd1af2c148b0535e7285217f97493fc1bf9241fbad915147528e96e0204a: Status 404 returned error can't find the container with id 5aa2dd1af2c148b0535e7285217f97493fc1bf9241fbad915147528e96e0204a Jul 08 09:09:02.769152 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.769113 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" event={"ID":"efd395c9-49c7-4b95-a7e6-c8d40d5cb19c","Type":"ContainerStarted","Data":"b37644fc4cd3ce40260987e61b0156b57e402ebcff5351aabcc40672561bc719"} Jul 08 09:09:02.770536 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.770506 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="05ab56675700a62a14aebb248424a6577cfbd33df2a38eff32da57c3b3d9a07f" exitCode=0 Jul 08 09:09:02.770672 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.770608 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"05ab56675700a62a14aebb248424a6577cfbd33df2a38eff32da57c3b3d9a07f"} Jul 08 09:09:02.770769 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.770674 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"5aa2dd1af2c148b0535e7285217f97493fc1bf9241fbad915147528e96e0204a"} Jul 08 09:09:02.774701 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.774675 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerStarted","Data":"182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c"} Jul 08 09:09:02.793164 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.793115 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" podStartSLOduration=1.731700536 podStartE2EDuration="3.793099811s" totalImagesPullingTime="2.061399275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:09:02.791789344 +0000 UTC m=+121.146800967" watchObservedRunningTime="2026-07-08 09:09:02.793099811 +0000 UTC m=+121.148111373" Jul 08 09:09:02.825631 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:02.825579 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.774185224 podStartE2EDuration="7.825562761s" totalImagesPullingTime="4.035418681s" totalInitContainerRuntime="15.958856ms" isStatefulPod=false podCreationTimestamp="2026-07-08 09:08:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:09:02.823859103 +0000 UTC m=+121.178870666" watchObservedRunningTime="2026-07-08 09:09:02.825562761 +0000 UTC m=+121.180574322" Jul 08 09:09:03.662430 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:03.662388 2572 patch_prober.go:28] interesting pod/image-registry-86b94ddf7f-xzhhk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 09:09:03.662851 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:03.662465 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" podUID="f785069b-d367-48c5-bfe2-53cc471b0142" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 09:09:03.836084 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:03.836035 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:03.903517 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:03.903480 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:05.673598 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:05.673570 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86f45cdf87-khsjs" Jul 08 09:09:05.826803 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:05.826770 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:05.928513 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:05.928421 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:07.690323 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:07.690281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:07.800808 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:07.800776 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"b89303dad0df4b69ca7fbd1f7c02fbdd7a32740a4fb8a6d01e39e1d3ab5a15e5"} Jul 08 09:09:07.800998 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:07.800883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"d94933d043d6a71f2f6787096219e45bf6001115c32b9f10012ec1af41208f92"} Jul 08 09:09:07.807822 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:07.807787 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:08.679410 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:08.679356 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" podUID="f785069b-d367-48c5-bfe2-53cc471b0142" containerName="registry" containerID="cri-o://6d4ffb10acaae3b0d887e55e186f0fa71df03f50da200cf7d61a594295e0450f" gracePeriod=30 Jul 08 09:09:08.806813 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:08.806784 2572 generic.go:356] "Generic (PLEG): container finished" podID="f785069b-d367-48c5-bfe2-53cc471b0142" containerID="6d4ffb10acaae3b0d887e55e186f0fa71df03f50da200cf7d61a594295e0450f" exitCode=0 Jul 08 09:09:08.807163 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:08.806842 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" event={"ID":"f785069b-d367-48c5-bfe2-53cc471b0142","Type":"ContainerDied","Data":"6d4ffb10acaae3b0d887e55e186f0fa71df03f50da200cf7d61a594295e0450f"} Jul 08 09:09:08.962282 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:08.962120 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:09:09.017231 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017198 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-registry-certificates\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017360 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017298 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f785069b-d367-48c5-bfe2-53cc471b0142-ca-trust-extracted\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017360 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017340 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-bound-sa-token\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017488 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017390 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-image-registry-private-configuration\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017488 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017414 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-trusted-ca\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017488 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017443 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhdbc\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-kube-api-access-lhdbc\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017488 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017478 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.017684 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.017503 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-installation-pull-secrets\") pod \"f785069b-d367-48c5-bfe2-53cc471b0142\" (UID: \"f785069b-d367-48c5-bfe2-53cc471b0142\") " Jul 08 09:09:09.024933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.022349 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:09:09.024933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.022613 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:09:09.024933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.022627 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:09:09.024933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.023420 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:09:09.024933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.023518 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:09:09.024933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.023612 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-kube-api-access-lhdbc" (OuterVolumeSpecName: "kube-api-access-lhdbc") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "kube-api-access-lhdbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:09:09.025355 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.025045 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:09:09.032152 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.031848 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f785069b-d367-48c5-bfe2-53cc471b0142-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f785069b-d367-48c5-bfe2-53cc471b0142" (UID: "f785069b-d367-48c5-bfe2-53cc471b0142"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:09:09.118949 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118890 2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f785069b-d367-48c5-bfe2-53cc471b0142-ca-trust-extracted\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.118949 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118945 2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-bound-sa-token\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.118949 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118956 2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-image-registry-private-configuration\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.119174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118965 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-trusted-ca\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.119174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118974 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhdbc\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-kube-api-access-lhdbc\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.119174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118982 2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f785069b-d367-48c5-bfe2-53cc471b0142-registry-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.119174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.118990 2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f785069b-d367-48c5-bfe2-53cc471b0142-installation-pull-secrets\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.119174 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.119006 2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f785069b-d367-48c5-bfe2-53cc471b0142-registry-certificates\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:09:09.814021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.813982 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"6df0a900cfbf9c9015b27f7cca54ca2dbde98576deae68ac4ae53ee90ac6499f"} Jul 08 09:09:09.814481 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.814077 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"65f866be82c5bee33c6d9da5f76332de08e05f9bdf852e343b1d1cc1545d8076"} Jul 08 09:09:09.814481 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.814108 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"c12098b0f152445e270fddcc8070935022b36d5e8fd80e22ad113ce539bb561a"} Jul 08 09:09:09.814481 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.814136 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerStarted","Data":"4a611afce1adaa315e8c51a7532e8bb8ae83e780e149d39799bf33f395520bc0"} Jul 08 09:09:09.815274 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.815231 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" event={"ID":"f785069b-d367-48c5-bfe2-53cc471b0142","Type":"ContainerDied","Data":"de66efbc6954bb7c75b0da323cb5e156a333fec9a7ca68687e07202235444717"} Jul 08 09:09:09.815395 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.815297 2572 scope.go:117] "RemoveContainer" containerID="6d4ffb10acaae3b0d887e55e186f0fa71df03f50da200cf7d61a594295e0450f" Jul 08 09:09:09.815395 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.815255 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86b94ddf7f-xzhhk" Jul 08 09:09:09.853210 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.853165 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.93985251 podStartE2EDuration="8.853148042s" totalImagesPullingTime="5.893870857s" totalInitContainerRuntime="19.424675ms" isStatefulPod=false podCreationTimestamp="2026-07-08 09:09:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:09:09.852052341 +0000 UTC m=+128.207063905" watchObservedRunningTime="2026-07-08 09:09:09.853148042 +0000 UTC m=+128.208159603" Jul 08 09:09:09.870422 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.870399 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86b94ddf7f-xzhhk"] Jul 08 09:09:09.878137 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:09.878114 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86b94ddf7f-xzhhk"] Jul 08 09:09:10.178612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:10.178579 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f785069b-d367-48c5-bfe2-53cc471b0142" path="/var/lib/kubelet/pods/f785069b-d367-48c5-bfe2-53cc471b0142/volumes" Jul 08 09:09:10.390263 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:10.390235 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:10.489824 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:10.489743 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:10.938033 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:10.937956 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:09:10.940480 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:10.940449 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c3d6d5b-04ee-420f-a078-8e63b08436e4-metrics-certs\") pod \"network-metrics-daemon-fgcz8\" (UID: \"8c3d6d5b-04ee-420f-a078-8e63b08436e4\") " pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:09:11.085828 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:11.085794 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6bnkm\"" Jul 08 09:09:11.093764 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:11.093744 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fgcz8" Jul 08 09:09:11.310643 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:11.310617 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fgcz8"] Jul 08 09:09:11.313243 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:09:11.313221 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3d6d5b_04ee_420f_a078_8e63b08436e4.slice/crio-c7921c163001e8b299d3fecde3aebe719f88a7fcb69a42ab7282cea31d0bce52 WatchSource:0}: Error finding container c7921c163001e8b299d3fecde3aebe719f88a7fcb69a42ab7282cea31d0bce52: Status 404 returned error can't find the container with id c7921c163001e8b299d3fecde3aebe719f88a7fcb69a42ab7282cea31d0bce52 Jul 08 09:09:11.540968 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:11.540875 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:09:11.825850 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:11.825773 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgcz8" event={"ID":"8c3d6d5b-04ee-420f-a078-8e63b08436e4","Type":"ContainerStarted","Data":"c7921c163001e8b299d3fecde3aebe719f88a7fcb69a42ab7282cea31d0bce52"} Jul 08 09:09:11.995350 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:11.995284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:12.096867 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.096785 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:12.346971 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.346870 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:12.414338 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.414301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:12.831348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.831314 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgcz8" event={"ID":"8c3d6d5b-04ee-420f-a078-8e63b08436e4","Type":"ContainerStarted","Data":"38d7f0878a755e1f0a8ea2fd0ed898b4a2777d92df30950bb674406492675e0e"} Jul 08 09:09:12.831506 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.831399 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fgcz8" event={"ID":"8c3d6d5b-04ee-420f-a078-8e63b08436e4","Type":"ContainerStarted","Data":"55ea1c60df39630f4dd0bbc6439e16b31afb97a1cb444f6689201dfe44c1b557"} Jul 08 09:09:12.847245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.847215 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:12.859269 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.859180 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fgcz8" podStartSLOduration=129.703414142 podStartE2EDuration="2m10.859165554s" totalImagesPullingTime="1.155751412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:09:12.857405084 +0000 UTC m=+131.212416648" watchObservedRunningTime="2026-07-08 09:09:12.859165554 +0000 UTC m=+131.214177117" Jul 08 09:09:12.967473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:12.967422 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:13.129805 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:13.129712 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:13.334975 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:13.334941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:14.422508 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:14.422473 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:14.630628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:14.630595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:16.037561 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:16.037525 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:16.160636 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:16.160602 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:18.071149 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:18.071114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:18.181844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:18.181813 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:19.614962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:19.614934 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:09:19.615318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:19.614995 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:09:20.083358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:20.083275 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:20.189223 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:20.189191 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:21.580040 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:21.580002 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:21.717520 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:21.717484 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:28.423050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:28.423010 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:28.523126 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:28.523089 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:29.297230 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:29.297193 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:31.908455 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:31.908418 2572 generic.go:356] "Generic (PLEG): container finished" podID="9f6e3800-8e80-411b-9506-d6f9f4042f0b" containerID="206e831b01fb9cda04b5080c6bed59f91ded6c342c1bec34ec03ebb3d40443e2" exitCode=0 Jul 08 09:09:31.908887 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:31.908493 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" event={"ID":"9f6e3800-8e80-411b-9506-d6f9f4042f0b","Type":"ContainerDied","Data":"206e831b01fb9cda04b5080c6bed59f91ded6c342c1bec34ec03ebb3d40443e2"} Jul 08 09:09:31.908887 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:31.908802 2572 scope.go:117] "RemoveContainer" containerID="206e831b01fb9cda04b5080c6bed59f91ded6c342c1bec34ec03ebb3d40443e2" Jul 08 09:09:32.914309 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:32.914274 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-kk6rv" event={"ID":"9f6e3800-8e80-411b-9506-d6f9f4042f0b","Type":"ContainerStarted","Data":"bfa466495d942766ee86cbbf50b0df347fa259a1486f9e8cc9747e81a20fdf00"} Jul 08 09:09:36.931713 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:36.931679 2572 generic.go:356] "Generic (PLEG): container finished" podID="048aa6fd-5839-43dd-a471-1f58a7ab8fd4" containerID="bc64f246d60c6a4b898f4cb067d3a8c87c80346967363af8d174e636face2e1b" exitCode=0 Jul 08 09:09:36.932121 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:36.931751 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" event={"ID":"048aa6fd-5839-43dd-a471-1f58a7ab8fd4","Type":"ContainerDied","Data":"bc64f246d60c6a4b898f4cb067d3a8c87c80346967363af8d174e636face2e1b"} Jul 08 09:09:36.932121 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:36.932074 2572 scope.go:117] "RemoveContainer" containerID="bc64f246d60c6a4b898f4cb067d3a8c87c80346967363af8d174e636face2e1b" Jul 08 09:09:37.938110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:37.938080 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qpjjd" event={"ID":"048aa6fd-5839-43dd-a471-1f58a7ab8fd4","Type":"ContainerStarted","Data":"78b98057174a76eeeb9cf10afe113716b55922e5c41c8e275d1638fb71656c3f"} Jul 08 09:09:39.619300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:39.619272 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:09:39.623070 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:39.623049 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8699564ddd-krr97" Jul 08 09:09:43.479650 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:43.479612 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:43.561326 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:43.561296 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:47.155260 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:47.155223 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:09:47.241116 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:09:47.241081 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:01.541193 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:01.541143 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:01.560936 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:01.560913 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:02.049314 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:02.049279 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:14.333628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:14.333594 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:14.707888 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:14.707831 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:15.323777 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.323744 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:10:15.324093 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.324070 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="alertmanager" containerID="cri-o://5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8" gracePeriod=120 Jul 08 09:10:15.324192 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.324134 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-metric" containerID="cri-o://e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538" gracePeriod=120 Jul 08 09:10:15.324354 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.324168 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy" containerID="cri-o://308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787" gracePeriod=120 Jul 08 09:10:15.324426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.324168 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-web" containerID="cri-o://a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7" gracePeriod=120 Jul 08 09:10:15.324492 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.324212 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="prom-label-proxy" containerID="cri-o://182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c" gracePeriod=120 Jul 08 09:10:15.325102 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:15.324208 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="config-reloader" containerID="cri-o://487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443" gracePeriod=120 Jul 08 09:10:16.102225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102189 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c" exitCode=0 Jul 08 09:10:16.102225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102217 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538" exitCode=0 Jul 08 09:10:16.102225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102225 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787" exitCode=0 Jul 08 09:10:16.102225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102235 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443" exitCode=0 Jul 08 09:10:16.102767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102243 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8" exitCode=0 Jul 08 09:10:16.102767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c"} Jul 08 09:10:16.102767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102377 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538"} Jul 08 09:10:16.102767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102421 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787"} Jul 08 09:10:16.102767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102461 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443"} Jul 08 09:10:16.102767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.102501 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8"} Jul 08 09:10:16.572323 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.572297 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:16.704228 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704139 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2xxd\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-kube-api-access-j2xxd\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704228 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704195 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-web\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704228 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704214 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704234 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-cluster-tls-config\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704368 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-volume\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704404 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-web-config\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704443 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-tls-assets\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704481 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-main-db\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704549 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-metrics-client-ca\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704616 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-main-tls\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704658 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-trusted-ca-bundle\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704732 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-out\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.704767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.704765 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\" (UID: \"470eb0c6-3326-478b-8d6d-008f39dd0ec9\") " Jul 08 09:10:16.705088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.705058 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:16.706208 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.705828 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:16.706208 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.706004 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:10:16.706991 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.706950 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.707284 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.707252 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.707542 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.707508 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.707636 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.707573 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-kube-api-access-j2xxd" (OuterVolumeSpecName: "kube-api-access-j2xxd") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "kube-api-access-j2xxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:10:16.707697 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.707662 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-volume" (OuterVolumeSpecName: "config-volume") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.707697 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.707683 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:10:16.708429 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.708400 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-out" (OuterVolumeSpecName: "config-out") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:10:16.709247 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.709222 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.713224 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.713128 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.718789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.718762 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-web-config" (OuterVolumeSpecName: "web-config") pod "470eb0c6-3326-478b-8d6d-008f39dd0ec9" (UID: "470eb0c6-3326-478b-8d6d-008f39dd0ec9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:16.806434 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806397 2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-out\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806434 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806427 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806441 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2xxd\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-kube-api-access-j2xxd\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806455 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806468 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806482 2572 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-cluster-tls-config\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806493 2572 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-config-volume\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806504 2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-web-config\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806514 2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/470eb0c6-3326-478b-8d6d-008f39dd0ec9-tls-assets\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806524 2572 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-main-db\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806535 2572 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-metrics-client-ca\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806547 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/470eb0c6-3326-478b-8d6d-008f39dd0ec9-secret-alertmanager-main-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:16.806638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:16.806560 2572 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/470eb0c6-3326-478b-8d6d-008f39dd0ec9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:17.108963 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.108932 2572 generic.go:356] "Generic (PLEG): container finished" podID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerID="a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7" exitCode=0 Jul 08 09:10:17.109325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.109010 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.109325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.109018 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7"} Jul 08 09:10:17.109325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.109095 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"470eb0c6-3326-478b-8d6d-008f39dd0ec9","Type":"ContainerDied","Data":"64594a6583267145e09456a49a228130ba6fac97c81b555674fe02e4d153ae81"} Jul 08 09:10:17.109325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.109132 2572 scope.go:117] "RemoveContainer" containerID="182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c" Jul 08 09:10:17.117737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.117723 2572 scope.go:117] "RemoveContainer" containerID="e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538" Jul 08 09:10:17.124638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.124622 2572 scope.go:117] "RemoveContainer" containerID="308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787" Jul 08 09:10:17.131342 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.131324 2572 scope.go:117] "RemoveContainer" containerID="a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7" Jul 08 09:10:17.140806 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.140251 2572 scope.go:117] "RemoveContainer" containerID="487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443" Jul 08 09:10:17.145659 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.145640 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:10:17.148772 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.148755 2572 scope.go:117] "RemoveContainer" containerID="5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8" Jul 08 09:10:17.156037 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.156013 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:10:17.156554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.156539 2572 scope.go:117] "RemoveContainer" containerID="fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4" Jul 08 09:10:17.163715 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.163698 2572 scope.go:117] "RemoveContainer" containerID="182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c" Jul 08 09:10:17.163984 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.163954 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c\": container with ID starting with 182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c not found: ID does not exist" containerID="182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c" Jul 08 09:10:17.164032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.163995 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c"} err="failed to get container status \"182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c\": rpc error: code = NotFound desc = could not find container \"182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c\": container with ID starting with 182cd61db9bea8894f807d91f04551e471dda934be0c2230bff67774e0de107c not found: ID does not exist" Jul 08 09:10:17.164032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164029 2572 scope.go:117] "RemoveContainer" containerID="e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538" Jul 08 09:10:17.164252 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.164236 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538\": container with ID starting with e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538 not found: ID does not exist" containerID="e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538" Jul 08 09:10:17.164291 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164260 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538"} err="failed to get container status \"e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538\": rpc error: code = NotFound desc = could not find container \"e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538\": container with ID starting with e1094603f46fc6c8087b18096100ea3524b4e3d42f6d25f4af52dd49625f4538 not found: ID does not exist" Jul 08 09:10:17.164291 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164276 2572 scope.go:117] "RemoveContainer" containerID="308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787" Jul 08 09:10:17.164495 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.164479 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787\": container with ID starting with 308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787 not found: ID does not exist" containerID="308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787" Jul 08 09:10:17.164534 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164499 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787"} err="failed to get container status \"308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787\": rpc error: code = NotFound desc = could not find container \"308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787\": container with ID starting with 308d779337e7664c9062a3886295b1bd7b7cac9c9e7364879bef03bf47df0787 not found: ID does not exist" Jul 08 09:10:17.164534 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164512 2572 scope.go:117] "RemoveContainer" containerID="a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7" Jul 08 09:10:17.164698 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.164677 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7\": container with ID starting with a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7 not found: ID does not exist" containerID="a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7" Jul 08 09:10:17.164733 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164706 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7"} err="failed to get container status \"a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7\": rpc error: code = NotFound desc = could not find container \"a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7\": container with ID starting with a17d68b8a0980e20ba2aa49522d41c4fe20cc5d8fe9b6545ac67a3535c4be4b7 not found: ID does not exist" Jul 08 09:10:17.164733 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164727 2572 scope.go:117] "RemoveContainer" containerID="487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443" Jul 08 09:10:17.164942 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.164923 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443\": container with ID starting with 487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443 not found: ID does not exist" containerID="487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443" Jul 08 09:10:17.165024 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164951 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443"} err="failed to get container status \"487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443\": rpc error: code = NotFound desc = could not find container \"487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443\": container with ID starting with 487fb7dcfe06f655eba468d731edc5e144d71fef6b60b53a08747944383f0443 not found: ID does not exist" Jul 08 09:10:17.165024 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.164971 2572 scope.go:117] "RemoveContainer" containerID="5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8" Jul 08 09:10:17.165245 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.165222 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8\": container with ID starting with 5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8 not found: ID does not exist" containerID="5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8" Jul 08 09:10:17.165293 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.165254 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8"} err="failed to get container status \"5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8\": rpc error: code = NotFound desc = could not find container \"5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8\": container with ID starting with 5b2d749f115c9ac9cfa9727c53055230f3621adf4056ee928a9b4afe295918c8 not found: ID does not exist" Jul 08 09:10:17.165293 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.165268 2572 scope.go:117] "RemoveContainer" containerID="fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4" Jul 08 09:10:17.165487 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:10:17.165461 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4\": container with ID starting with fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4 not found: ID does not exist" containerID="fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4" Jul 08 09:10:17.165532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.165482 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4"} err="failed to get container status \"fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4\": rpc error: code = NotFound desc = could not find container \"fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4\": container with ID starting with fc8fe3b86c564115b022d2f9ac858b439306fcc0a879f12c66dfbed56becded4 not found: ID does not exist" Jul 08 09:10:17.358739 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.358711 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:10:17.359078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359038 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-web" Jul 08 09:10:17.359078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359054 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-web" Jul 08 09:10:17.359078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359063 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="prom-label-proxy" Jul 08 09:10:17.359078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359069 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="prom-label-proxy" Jul 08 09:10:17.359078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359079 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="alertmanager" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359085 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="alertmanager" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359091 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f785069b-d367-48c5-bfe2-53cc471b0142" containerName="registry" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359097 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f785069b-d367-48c5-bfe2-53cc471b0142" containerName="registry" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359106 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="init-config-reloader" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359111 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="init-config-reloader" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359117 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-metric" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359122 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-metric" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359132 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="config-reloader" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359138 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="config-reloader" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359143 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359148 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359193 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="alertmanager" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359200 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-metric" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359207 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="config-reloader" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359213 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f785069b-d367-48c5-bfe2-53cc471b0142" containerName="registry" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359219 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy-web" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359226 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="prom-label-proxy" Jul 08 09:10:17.359250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.359234 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" containerName="kube-rbac-proxy" Jul 08 09:10:17.380831 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.380805 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.388038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.388016 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 09:10:17.390043 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.390019 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:10:17.403369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.403349 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 09:10:17.414802 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.414782 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 09:10:17.417546 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.417525 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 09:10:17.417699 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.417682 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 09:10:17.418861 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.418845 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 09:10:17.442289 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.442251 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 09:10:17.442289 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.442259 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 09:10:17.442582 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.442567 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t7pbl\"" Jul 08 09:10:17.468925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.468872 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 09:10:17.511242 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511247 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511269 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511350 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-config-out\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511373 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2cfb\" (UniqueName: \"kubernetes.io/projected/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-kube-api-access-h2cfb\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511420 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511585 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511491 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511585 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-web-config\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511585 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511542 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511585 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511585 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.511739 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.511613 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.612991 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613046 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613240 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613280 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613341 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-config-out\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613618 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613372 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.613667 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.613612 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.614616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.614057 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.614616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.614111 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2cfb\" (UniqueName: \"kubernetes.io/projected/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-kube-api-access-h2cfb\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.614616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.614173 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.614616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.614215 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.614616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.614241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-web-config\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.614616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.614286 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.616231 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.616207 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.616410 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.616353 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-config-out\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.616538 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.616516 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.616616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.616520 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.616801 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.616782 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.616927 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.616840 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.617024 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.617001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.617678 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.617652 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-web-config\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.618130 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.618110 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.632476 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.632455 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2cfb\" (UniqueName: \"kubernetes.io/projected/2a4c6d5d-8922-4c81-81f0-fc38dad66bad-kube-api-access-h2cfb\") pod \"alertmanager-main-0\" (UID: \"2a4c6d5d-8922-4c81-81f0-fc38dad66bad\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.692445 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.692402 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 09:10:17.847758 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:17.847733 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 09:10:17.850334 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:10:17.850295 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4c6d5d_8922_4c81_81f0_fc38dad66bad.slice/crio-171e53583eaf3eee6f2bd7d3c5a648840de74c086fad2f61512b0891fc46d00e WatchSource:0}: Error finding container 171e53583eaf3eee6f2bd7d3c5a648840de74c086fad2f61512b0891fc46d00e: Status 404 returned error can't find the container with id 171e53583eaf3eee6f2bd7d3c5a648840de74c086fad2f61512b0891fc46d00e Jul 08 09:10:18.115741 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:18.115704 2572 generic.go:356] "Generic (PLEG): container finished" podID="2a4c6d5d-8922-4c81-81f0-fc38dad66bad" containerID="58078d38e4f1867759c7349d2e710a9ba4cbbba7e233c7bec46ec6e3c6f8f041" exitCode=0 Jul 08 09:10:18.116101 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:18.115759 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerDied","Data":"58078d38e4f1867759c7349d2e710a9ba4cbbba7e233c7bec46ec6e3c6f8f041"} Jul 08 09:10:18.116101 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:18.115825 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"171e53583eaf3eee6f2bd7d3c5a648840de74c086fad2f61512b0891fc46d00e"} Jul 08 09:10:18.174692 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:18.174658 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="470eb0c6-3326-478b-8d6d-008f39dd0ec9" path="/var/lib/kubelet/pods/470eb0c6-3326-478b-8d6d-008f39dd0ec9/volumes" Jul 08 09:10:19.123120 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.123084 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"818b6b63d596c4f5921c41cf389d48369fe4c467f57a55e3fdc77610042aa8de"} Jul 08 09:10:19.123563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.123150 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"29e357306ed98f1032edc9b4e17b4bce8ac1f3bb791ac4f5971a2e13d1b55c49"} Jul 08 09:10:19.123563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.123177 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"c3101ea6b4ad586539f2d51ed467746a5aee2d7f6e68d87476fe4dcffbf0b82e"} Jul 08 09:10:19.123563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.123202 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"c13c5e91027d7fa16cf8a9a060226f1011977964041d605afb89b85d08f083db"} Jul 08 09:10:19.123563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.123225 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"1e5aa57e5ee8ee7d93bcb1122f4437306783efecfbbecfd14d77b1f6759ce102"} Jul 08 09:10:19.123563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.123247 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a4c6d5d-8922-4c81-81f0-fc38dad66bad","Type":"ContainerStarted","Data":"68c144f2b86ecdae899cc7d81d6c6f6fe9043619309385541db49d38fa0b33e7"} Jul 08 09:10:19.157505 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.157446 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.142165075 podStartE2EDuration="2.157427024s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.261949ms" isStatefulPod=false podCreationTimestamp="2026-07-08 09:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:10:19.157322855 +0000 UTC m=+197.512334417" watchObservedRunningTime="2026-07-08 09:10:19.157427024 +0000 UTC m=+197.512438587" Jul 08 09:10:19.865278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865245 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:10:19.865554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865529 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="prometheus" containerID="cri-o://d94933d043d6a71f2f6787096219e45bf6001115c32b9f10012ec1af41208f92" gracePeriod=600 Jul 08 09:10:19.865628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865567 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="thanos-sidecar" containerID="cri-o://4a611afce1adaa315e8c51a7532e8bb8ae83e780e149d39799bf33f395520bc0" gracePeriod=600 Jul 08 09:10:19.865628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865593 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="config-reloader" containerID="cri-o://b89303dad0df4b69ca7fbd1f7c02fbdd7a32740a4fb8a6d01e39e1d3ab5a15e5" gracePeriod=600 Jul 08 09:10:19.865628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865583 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy" containerID="cri-o://65f866be82c5bee33c6d9da5f76332de08e05f9bdf852e343b1d1cc1545d8076" gracePeriod=600 Jul 08 09:10:19.865794 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865697 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-web" containerID="cri-o://c12098b0f152445e270fddcc8070935022b36d5e8fd80e22ad113ce539bb561a" gracePeriod=600 Jul 08 09:10:19.865794 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:19.865727 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6df0a900cfbf9c9015b27f7cca54ca2dbde98576deae68ac4ae53ee90ac6499f" gracePeriod=600 Jul 08 09:10:20.134657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134626 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="6df0a900cfbf9c9015b27f7cca54ca2dbde98576deae68ac4ae53ee90ac6499f" exitCode=0 Jul 08 09:10:20.134657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134651 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="65f866be82c5bee33c6d9da5f76332de08e05f9bdf852e343b1d1cc1545d8076" exitCode=0 Jul 08 09:10:20.134657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134657 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="c12098b0f152445e270fddcc8070935022b36d5e8fd80e22ad113ce539bb561a" exitCode=0 Jul 08 09:10:20.134657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134663 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="4a611afce1adaa315e8c51a7532e8bb8ae83e780e149d39799bf33f395520bc0" exitCode=0 Jul 08 09:10:20.134657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134668 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="b89303dad0df4b69ca7fbd1f7c02fbdd7a32740a4fb8a6d01e39e1d3ab5a15e5" exitCode=0 Jul 08 09:10:20.134657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134677 2572 generic.go:356] "Generic (PLEG): container finished" podID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerID="d94933d043d6a71f2f6787096219e45bf6001115c32b9f10012ec1af41208f92" exitCode=0 Jul 08 09:10:20.135369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134699 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"6df0a900cfbf9c9015b27f7cca54ca2dbde98576deae68ac4ae53ee90ac6499f"} Jul 08 09:10:20.135369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134782 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"65f866be82c5bee33c6d9da5f76332de08e05f9bdf852e343b1d1cc1545d8076"} Jul 08 09:10:20.135369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134827 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"c12098b0f152445e270fddcc8070935022b36d5e8fd80e22ad113ce539bb561a"} Jul 08 09:10:20.135369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134865 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"4a611afce1adaa315e8c51a7532e8bb8ae83e780e149d39799bf33f395520bc0"} Jul 08 09:10:20.135369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134931 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"b89303dad0df4b69ca7fbd1f7c02fbdd7a32740a4fb8a6d01e39e1d3ab5a15e5"} Jul 08 09:10:20.135369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.134975 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"d94933d043d6a71f2f6787096219e45bf6001115c32b9f10012ec1af41208f92"} Jul 08 09:10:20.138535 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.138502 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:20.238005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.237976 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-metrics-client-certs\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238150 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238022 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-metrics-client-ca\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238150 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238054 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw729\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-kube-api-access-kw729\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238282 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238247 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-web-config\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238345 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238296 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-1\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238345 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238332 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238365 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-grpc-tls\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238408 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-2\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238447 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238439 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-db\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238445 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.238609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238461 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-tls-assets\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238490 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-trusted-ca-bundle\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238540 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-thanos-prometheus-http-client-file\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238574 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238601 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-tls\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238625 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-config\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238664 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-kube-rbac-proxy\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238701 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-config-out\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238737 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-0\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238769 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-serving-certs-ca-bundle\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.238944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.238800 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-kubelet-serving-ca-bundle\") pod \"8bb9c7e1-edb4-4e36-88ab-db418428e340\" (UID: \"8bb9c7e1-edb4-4e36-88ab-db418428e340\") " Jul 08 09:10:20.240335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.239326 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-metrics-client-ca\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.240335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.239596 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.240335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.239861 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:10:20.240335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.240006 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.240335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.240015 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.240853 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.240522 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.240853 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.240805 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.241580 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.241297 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.242265 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.242234 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.242704 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.242442 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:10:20.242854 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.242827 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-kube-api-access-kw729" (OuterVolumeSpecName: "kube-api-access-kw729") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "kube-api-access-kw729". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:10:20.243324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.243247 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.243324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.243291 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.243473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.243336 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:10:20.243473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.243342 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.244107 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.244082 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-config-out" (OuterVolumeSpecName: "config-out") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:10:20.244218 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.244180 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.244792 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.244767 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-config" (OuterVolumeSpecName: "config") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.244857 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.244842 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.253205 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.253185 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-web-config" (OuterVolumeSpecName: "web-config") pod "8bb9c7e1-edb4-4e36-88ab-db418428e340" (UID: "8bb9c7e1-edb4-4e36-88ab-db418428e340"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:10:20.340689 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340653 2572 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340689 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340681 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340689 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340693 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340702 2572 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-config\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340711 2572 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-kube-rbac-proxy\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340719 2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-config-out\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340727 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340735 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340745 2572 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340753 2572 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-metrics-client-certs\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340763 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw729\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-kube-api-access-kw729\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340772 2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-web-config\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340782 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340789 2572 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340797 2572 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8bb9c7e1-edb4-4e36-88ab-db418428e340-secret-grpc-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340806 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340814 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-k8s-db\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340824 2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb9c7e1-edb4-4e36-88ab-db418428e340-tls-assets\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:20.340929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:20.340832 2572 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb9c7e1-edb4-4e36-88ab-db418428e340-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:10:21.142018 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.141924 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8bb9c7e1-edb4-4e36-88ab-db418428e340","Type":"ContainerDied","Data":"5aa2dd1af2c148b0535e7285217f97493fc1bf9241fbad915147528e96e0204a"} Jul 08 09:10:21.142425 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.141940 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.142425 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.142023 2572 scope.go:117] "RemoveContainer" containerID="6df0a900cfbf9c9015b27f7cca54ca2dbde98576deae68ac4ae53ee90ac6499f" Jul 08 09:10:21.152994 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.152957 2572 scope.go:117] "RemoveContainer" containerID="65f866be82c5bee33c6d9da5f76332de08e05f9bdf852e343b1d1cc1545d8076" Jul 08 09:10:21.160469 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.160451 2572 scope.go:117] "RemoveContainer" containerID="c12098b0f152445e270fddcc8070935022b36d5e8fd80e22ad113ce539bb561a" Jul 08 09:10:21.167058 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.167041 2572 scope.go:117] "RemoveContainer" containerID="4a611afce1adaa315e8c51a7532e8bb8ae83e780e149d39799bf33f395520bc0" Jul 08 09:10:21.174090 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.173943 2572 scope.go:117] "RemoveContainer" containerID="b89303dad0df4b69ca7fbd1f7c02fbdd7a32740a4fb8a6d01e39e1d3ab5a15e5" Jul 08 09:10:21.174849 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.174828 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:10:21.180532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.180508 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:10:21.182059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.182044 2572 scope.go:117] "RemoveContainer" containerID="d94933d043d6a71f2f6787096219e45bf6001115c32b9f10012ec1af41208f92" Jul 08 09:10:21.188988 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.188973 2572 scope.go:117] "RemoveContainer" containerID="05ab56675700a62a14aebb248424a6577cfbd33df2a38eff32da57c3b3d9a07f" Jul 08 09:10:21.211125 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211102 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:10:21.211431 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211417 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="config-reloader" Jul 08 09:10:21.211472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211434 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="config-reloader" Jul 08 09:10:21.211472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211442 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy" Jul 08 09:10:21.211472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211448 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy" Jul 08 09:10:21.211472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211459 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="init-config-reloader" Jul 08 09:10:21.211472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211465 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="init-config-reloader" Jul 08 09:10:21.211472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211471 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="prometheus" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211476 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="prometheus" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211482 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-thanos" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211488 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-thanos" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211500 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="thanos-sidecar" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211506 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="thanos-sidecar" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211520 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-web" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211525 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-web" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211579 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="prometheus" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211590 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-thanos" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211597 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="thanos-sidecar" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211602 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy-web" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211610 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="kube-rbac-proxy" Jul 08 09:10:21.211639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.211618 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" containerName="config-reloader" Jul 08 09:10:21.215628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.215611 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.218759 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.218741 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 09:10:21.219001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.218983 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 09:10:21.219082 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219001 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 09:10:21.219241 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219223 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 09:10:21.219320 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219274 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 09:10:21.219410 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219396 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xbl7k\"" Jul 08 09:10:21.219484 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219468 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 09:10:21.219545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219489 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 09:10:21.219545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219518 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 09:10:21.219784 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.219559 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 09:10:21.220080 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.220061 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fi8bge120ki7h\"" Jul 08 09:10:21.220202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.220182 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 09:10:21.220341 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.220186 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 09:10:21.220341 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.220338 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 09:10:21.224633 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.224617 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 09:10:21.228130 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.228114 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 09:10:21.234538 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.234515 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:10:21.248911 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.248885 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.248940 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.248968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nczpf\" (UniqueName: \"kubernetes.io/projected/1718182c-492c-48f4-ae5c-38c3ce460403-kube-api-access-nczpf\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249003 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249030 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249055 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249253 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249143 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249253 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249185 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-config\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249253 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249218 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249253 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249242 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1718182c-492c-48f4-ae5c-38c3ce460403-config-out\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249374 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249270 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249374 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249374 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249341 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249384 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249412 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249459 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-web-config\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249484 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249687 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1718182c-492c-48f4-ae5c-38c3ce460403-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.249687 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.249547 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.349876 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.349841 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1718182c-492c-48f4-ae5c-38c3ce460403-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.349876 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.349880 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350092 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.349929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350092 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.349973 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350092 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.349997 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nczpf\" (UniqueName: \"kubernetes.io/projected/1718182c-492c-48f4-ae5c-38c3ce460403-kube-api-access-nczpf\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350197 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350174 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350217 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350294 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350256 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350326 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350433 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350354 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-config\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350433 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350386 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350433 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350419 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1718182c-492c-48f4-ae5c-38c3ce460403-config-out\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350557 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350445 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350557 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350485 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350557 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350567 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350598 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350647 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-web-config\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350675 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350995 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350782 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.350995 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.350837 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.351181 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.351155 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.351632 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.351331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.351744 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.351693 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.352697 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.352671 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.353343 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.353317 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.353472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.353422 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.353980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.353961 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-config\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.354468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.354425 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.354560 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.354469 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1718182c-492c-48f4-ae5c-38c3ce460403-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.354842 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.354813 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.355066 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.355046 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.355260 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.355229 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1718182c-492c-48f4-ae5c-38c3ce460403-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.355730 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.355697 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1718182c-492c-48f4-ae5c-38c3ce460403-config-out\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.356044 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.356025 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.356136 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.356085 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.356434 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.356415 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-web-config\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.356992 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.356975 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1718182c-492c-48f4-ae5c-38c3ce460403-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.359246 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.359229 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nczpf\" (UniqueName: \"kubernetes.io/projected/1718182c-492c-48f4-ae5c-38c3ce460403-kube-api-access-nczpf\") pod \"prometheus-k8s-0\" (UID: \"1718182c-492c-48f4-ae5c-38c3ce460403\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.526196 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.526104 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:21.670065 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:21.670032 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 09:10:21.673086 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:10:21.673051 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1718182c_492c_48f4_ae5c_38c3ce460403.slice/crio-49112b799e5d472f6621efbf6128e9556ce2beb1aae56d0e8ba4ab2b0186bec8 WatchSource:0}: Error finding container 49112b799e5d472f6621efbf6128e9556ce2beb1aae56d0e8ba4ab2b0186bec8: Status 404 returned error can't find the container with id 49112b799e5d472f6621efbf6128e9556ce2beb1aae56d0e8ba4ab2b0186bec8 Jul 08 09:10:22.147632 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:22.147594 2572 generic.go:356] "Generic (PLEG): container finished" podID="1718182c-492c-48f4-ae5c-38c3ce460403" containerID="f6333f93d545a5ebaf888a038dfa376450daaed57f81ab967c5705d2f253912f" exitCode=0 Jul 08 09:10:22.148020 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:22.147651 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerDied","Data":"f6333f93d545a5ebaf888a038dfa376450daaed57f81ab967c5705d2f253912f"} Jul 08 09:10:22.148020 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:22.147704 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"49112b799e5d472f6621efbf6128e9556ce2beb1aae56d0e8ba4ab2b0186bec8"} Jul 08 09:10:22.175761 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:22.175729 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bb9c7e1-edb4-4e36-88ab-db418428e340" path="/var/lib/kubelet/pods/8bb9c7e1-edb4-4e36-88ab-db418428e340/volumes" Jul 08 09:10:23.154962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.154921 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"6b0d4aecd07edf57eb0b19d973c74fd9b652a46bee7fe311f8324ba6bf465272"} Jul 08 09:10:23.155336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.155041 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"85aa4d42c81cb61115075295fd5507799eddd58f76419f8b0756ed35e427762a"} Jul 08 09:10:23.155336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.155091 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"d1deba9d6374c2806d63f2e6c2dff01fe8412ed9fc4007da95c8d9d30ef9da2e"} Jul 08 09:10:23.155336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.155129 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"1052ff93dd20629c92788d538615be6db613faffe6d3ce604b0cbdad8285bba0"} Jul 08 09:10:23.155336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.155167 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"372c8c1af2d3d07921066e396835cbdf08fb52acde9bb23231693552c76577ba"} Jul 08 09:10:23.155336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.155203 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1718182c-492c-48f4-ae5c-38c3ce460403","Type":"ContainerStarted","Data":"6ccac54ee49d0ea78808d547c7661437a242e27f7010cd1668d3160c15c612e9"} Jul 08 09:10:23.187651 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:23.187593 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.167983799 podStartE2EDuration="2.187578457s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.594658ms" isStatefulPod=false podCreationTimestamp="2026-07-08 09:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:10:23.186331914 +0000 UTC m=+201.541343477" watchObservedRunningTime="2026-07-08 09:10:23.187578457 +0000 UTC m=+201.542590019" Jul 08 09:10:25.864561 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:25.864518 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:25.934493 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:25.934439 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:26.009796 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:26.009759 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:26.527236 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:26.527202 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:10:26.628770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:26.628735 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:26.778822 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:26.778722 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:32.978149 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:32.978110 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:33.041240 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:33.041212 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:36.444236 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:36.444189 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:36.544909 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:36.544871 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:10:36.612000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:10:36.611958 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:21.526677 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:21.526642 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:11:21.542308 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:21.542283 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:11:22.416443 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:22.416417 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 09:11:29.602334 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:29.602251 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:29.704152 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:29.704115 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:41.604988 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:41.604947 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:41.681594 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:41.681556 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:54.001044 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:54.001002 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:54.108693 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:54.108651 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:58.631479 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:58.631439 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:11:58.733726 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:11:58.733689 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:12:02.016877 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:12:02.016853 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 09:12:04.635479 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:12:04.635442 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:12:04.992225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:12:04.992129 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:12:30.336289 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:12:30.336253 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:12:30.424113 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:12:30.424077 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:12:58.704840 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:12:58.704804 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:00.645314 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:00.645275 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:02.387878 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:02.387837 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:03.511678 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:03.511639 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:03.685995 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:03.685955 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:05.907274 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:05.907244 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:06.109000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:06.108965 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:06.275462 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:06.275372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:06.423984 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:06.423946 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:08.398857 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:08.398820 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:08.579162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:08.579132 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:10.223686 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:10.223656 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:10.332647 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:10.332618 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:15.468987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:15.468952 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:17.375250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:17.375208 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:17.480665 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:17.480625 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:18.532366 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.532335 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56"] Jul 08 09:13:18.535804 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.535786 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.538329 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.538310 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jdzg8\"" Jul 08 09:13:18.539280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.539256 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 09:13:18.539498 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.539484 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 09:13:18.545667 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.545645 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56"] Jul 08 09:13:18.640527 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.640493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gggq4\" (UniqueName: \"kubernetes.io/projected/98803f93-c6cb-4f92-b0ef-e9de44fcbdfa-kube-api-access-gggq4\") pod \"openshift-lws-operator-fd8ccff4c-tkd56\" (UID: \"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.640691 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.640600 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98803f93-c6cb-4f92-b0ef-e9de44fcbdfa-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tkd56\" (UID: \"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.741670 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.741635 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98803f93-c6cb-4f92-b0ef-e9de44fcbdfa-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tkd56\" (UID: \"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.741823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.741700 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gggq4\" (UniqueName: \"kubernetes.io/projected/98803f93-c6cb-4f92-b0ef-e9de44fcbdfa-kube-api-access-gggq4\") pod \"openshift-lws-operator-fd8ccff4c-tkd56\" (UID: \"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.742045 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.742028 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98803f93-c6cb-4f92-b0ef-e9de44fcbdfa-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tkd56\" (UID: \"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.751500 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.751472 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gggq4\" (UniqueName: \"kubernetes.io/projected/98803f93-c6cb-4f92-b0ef-e9de44fcbdfa-kube-api-access-gggq4\") pod \"openshift-lws-operator-fd8ccff4c-tkd56\" (UID: \"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.856429 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.856409 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" Jul 08 09:13:18.996701 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:18.996676 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56"] Jul 08 09:13:18.999493 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:13:18.999466 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98803f93_c6cb_4f92_b0ef_e9de44fcbdfa.slice/crio-cf70e681366b2ceddfdbba4398df71ea719882536cee8c33ad0d57ef53274fa1 WatchSource:0}: Error finding container cf70e681366b2ceddfdbba4398df71ea719882536cee8c33ad0d57ef53274fa1: Status 404 returned error can't find the container with id cf70e681366b2ceddfdbba4398df71ea719882536cee8c33ad0d57ef53274fa1 Jul 08 09:13:19.001401 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:19.001385 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:13:19.855940 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:19.855860 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" event={"ID":"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa","Type":"ContainerStarted","Data":"cf70e681366b2ceddfdbba4398df71ea719882536cee8c33ad0d57ef53274fa1"} Jul 08 09:13:21.208140 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:21.208046 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:21.867250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:21.867219 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" event={"ID":"98803f93-c6cb-4f92-b0ef-e9de44fcbdfa","Type":"ContainerStarted","Data":"fc1cf18b1b41dc80fe46a1bc6338d6390618db04997f572488309672e408bc07"} Jul 08 09:13:21.889713 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:21.889664 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tkd56" podStartSLOduration=1.4515961800000001 podStartE2EDuration="3.88964917s" totalImagesPullingTime="2.43805299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:13:21.887494463 +0000 UTC m=+380.242506015" watchObservedRunningTime="2026-07-08 09:13:21.88964917 +0000 UTC m=+380.244660732" Jul 08 09:13:22.621826 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:22.621770 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:22.720658 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:22.720616 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:24.646789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:24.646747 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:35.856986 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:35.856949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:35.956168 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:35.956109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:36.543676 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:36.543639 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:36.651273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:36.651240 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:37.115962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.115926 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:37.624148 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.624104 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:37.726832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.726802 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:37.803654 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.803616 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp"] Jul 08 09:13:37.807025 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.807008 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:37.811114 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.811094 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 09:13:37.811700 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.811685 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 09:13:37.821468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.821448 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-j975l\"" Jul 08 09:13:37.821596 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.821579 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 09:13:37.822717 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.822699 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 09:13:37.858686 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.858663 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp"] Jul 08 09:13:37.906231 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.906144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c08bd53d-eb89-4df8-9e20-2961dc1f3034-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:37.906231 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.906182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c08bd53d-eb89-4df8-9e20-2961dc1f3034-manifests\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:37.906231 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.906209 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4lvd\" (UniqueName: \"kubernetes.io/projected/c08bd53d-eb89-4df8-9e20-2961dc1f3034-kube-api-access-t4lvd\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:37.906427 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.906360 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c08bd53d-eb89-4df8-9e20-2961dc1f3034-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:37.906427 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:37.906406 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c08bd53d-eb89-4df8-9e20-2961dc1f3034-tmp\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.007258 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.007229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c08bd53d-eb89-4df8-9e20-2961dc1f3034-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.007428 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.007270 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c08bd53d-eb89-4df8-9e20-2961dc1f3034-tmp\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.007428 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.007324 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c08bd53d-eb89-4df8-9e20-2961dc1f3034-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.007428 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.007346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c08bd53d-eb89-4df8-9e20-2961dc1f3034-manifests\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.007428 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.007366 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4lvd\" (UniqueName: \"kubernetes.io/projected/c08bd53d-eb89-4df8-9e20-2961dc1f3034-kube-api-access-t4lvd\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.007838 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.007809 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c08bd53d-eb89-4df8-9e20-2961dc1f3034-manifests\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.009591 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.009563 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c08bd53d-eb89-4df8-9e20-2961dc1f3034-tmp\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.009712 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.009693 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c08bd53d-eb89-4df8-9e20-2961dc1f3034-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.009785 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.009768 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c08bd53d-eb89-4df8-9e20-2961dc1f3034-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.023435 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.023406 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4lvd\" (UniqueName: \"kubernetes.io/projected/c08bd53d-eb89-4df8-9e20-2961dc1f3034-kube-api-access-t4lvd\") pod \"opendatahub-operator-controller-manager-66d6485f98-qg4hp\" (UID: \"c08bd53d-eb89-4df8-9e20-2961dc1f3034\") " pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.117656 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.117628 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:38.193300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.193263 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:38.301002 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.300868 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp"] Jul 08 09:13:38.306415 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:13:38.306388 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc08bd53d_eb89_4df8_9e20_2961dc1f3034.slice/crio-39e87dea04a73067e1357bef5ec273d0fcf0dc370be00b86020bd035b952e369 WatchSource:0}: Error finding container 39e87dea04a73067e1357bef5ec273d0fcf0dc370be00b86020bd035b952e369: Status 404 returned error can't find the container with id 39e87dea04a73067e1357bef5ec273d0fcf0dc370be00b86020bd035b952e369 Jul 08 09:13:38.338551 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.338505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:38.461730 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.461637 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:38.569482 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.569449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:38.704091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.704055 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:38.945467 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:38.945384 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" event={"ID":"c08bd53d-eb89-4df8-9e20-2961dc1f3034","Type":"ContainerStarted","Data":"39e87dea04a73067e1357bef5ec273d0fcf0dc370be00b86020bd035b952e369"} Jul 08 09:13:41.962070 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:41.962033 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" event={"ID":"c08bd53d-eb89-4df8-9e20-2961dc1f3034","Type":"ContainerStarted","Data":"4950e927e5f1a0f0d103611faa7e9634e7ad8ea2142ed441fe2792cdc03891ac"} Jul 08 09:13:42.967916 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:42.967866 2572 generic.go:356] "Generic (PLEG): container finished" podID="c08bd53d-eb89-4df8-9e20-2961dc1f3034" containerID="4950e927e5f1a0f0d103611faa7e9634e7ad8ea2142ed441fe2792cdc03891ac" exitCode=0 Jul 08 09:13:42.968283 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:42.967961 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" event={"ID":"c08bd53d-eb89-4df8-9e20-2961dc1f3034","Type":"ContainerDied","Data":"4950e927e5f1a0f0d103611faa7e9634e7ad8ea2142ed441fe2792cdc03891ac"} Jul 08 09:13:43.974255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:43.974222 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" event={"ID":"c08bd53d-eb89-4df8-9e20-2961dc1f3034","Type":"ContainerStarted","Data":"f61d40b181cb193ffe67328cf585c07c716983ea71fa4ff170d9f4ed3af17e62"} Jul 08 09:13:43.974635 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:43.974357 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:44.810888 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:44.810850 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:44.921735 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:44.921696 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:54.981805 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:54.981774 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" Jul 08 09:13:55.012310 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:55.012254 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66d6485f98-qg4hp" podStartSLOduration=13.220790085 podStartE2EDuration="18.012235864s" totalImagesPullingTime="3.704304455s" totalInitContainerRuntime="1.087141324s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:13:44.001346359 +0000 UTC m=+402.356357923" watchObservedRunningTime="2026-07-08 09:13:55.012235864 +0000 UTC m=+413.367247426" Jul 08 09:13:56.875650 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:56.875610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:57.162362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.162272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:57.745611 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.745580 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-x9c7p"] Jul 08 09:13:57.753211 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.753192 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:57.756561 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.756535 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bsw9s\"" Jul 08 09:13:57.756668 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.756583 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 09:13:57.788562 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.788529 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-x9c7p"] Jul 08 09:13:57.790160 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.790139 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8503c860-2306-4b4c-9b33-f0c86e69d98f-cert\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:57.790265 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.790172 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9srxv\" (UniqueName: \"kubernetes.io/projected/8503c860-2306-4b4c-9b33-f0c86e69d98f-kube-api-access-9srxv\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:57.856638 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.856603 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:57.891079 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.891048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8503c860-2306-4b4c-9b33-f0c86e69d98f-cert\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:57.891460 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.891086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9srxv\" (UniqueName: \"kubernetes.io/projected/8503c860-2306-4b4c-9b33-f0c86e69d98f-kube-api-access-9srxv\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:57.891460 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:13:57.891225 2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 09:13:57.891460 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:13:57.891289 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8503c860-2306-4b4c-9b33-f0c86e69d98f-cert podName:8503c860-2306-4b4c-9b33-f0c86e69d98f nodeName:}" failed. No retries permitted until 2026-07-08 09:13:58.391270476 +0000 UTC m=+416.746282022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8503c860-2306-4b4c-9b33-f0c86e69d98f-cert") pod "odh-model-controller-7998c67dc4-x9c7p" (UID: "8503c860-2306-4b4c-9b33-f0c86e69d98f") : secret "odh-model-controller-webhook-cert" not found Jul 08 09:13:57.943100 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:57.943071 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9srxv\" (UniqueName: \"kubernetes.io/projected/8503c860-2306-4b4c-9b33-f0c86e69d98f-kube-api-access-9srxv\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:58.137347 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.137306 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:58.333348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.333318 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:58.340493 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.340471 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5458fdb769-sv77r"] Jul 08 09:13:58.342742 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.342728 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.351019 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.350998 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 09:13:58.351102 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.350998 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 09:13:58.351398 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.351376 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 09:13:58.351466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.351421 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 09:13:58.371187 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.371159 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5458fdb769-sv77r"] Jul 08 09:13:58.396475 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.396409 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8503c860-2306-4b4c-9b33-f0c86e69d98f-cert\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:58.396475 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.396456 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-tls-certs\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.396668 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.396481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-tmp\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.396668 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.396609 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcs7d\" (UniqueName: \"kubernetes.io/projected/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-kube-api-access-vcs7d\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.398691 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.398665 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8503c860-2306-4b4c-9b33-f0c86e69d98f-cert\") pod \"odh-model-controller-7998c67dc4-x9c7p\" (UID: \"8503c860-2306-4b4c-9b33-f0c86e69d98f\") " pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:58.459082 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.459053 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:58.497413 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.497379 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-tls-certs\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.497563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.497422 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-tmp\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.497563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.497467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcs7d\" (UniqueName: \"kubernetes.io/projected/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-kube-api-access-vcs7d\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.499647 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.499624 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-tmp\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.499837 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.499820 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-tls-certs\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.514432 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.514407 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcs7d\" (UniqueName: \"kubernetes.io/projected/d12c84a8-3768-4dbe-8fa3-1b58d3c9f823-kube-api-access-vcs7d\") pod \"kube-auth-proxy-5458fdb769-sv77r\" (UID: \"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823\") " pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.653312 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.653207 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" Jul 08 09:13:58.663292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.663267 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:13:58.794417 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.794333 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5458fdb769-sv77r"] Jul 08 09:13:58.796976 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:13:58.796945 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12c84a8_3768_4dbe_8fa3_1b58d3c9f823.slice/crio-e572b40dfafd64f4ff9f4843ce4af118712fca2c679adb6caadf17c0783d0430 WatchSource:0}: Error finding container e572b40dfafd64f4ff9f4843ce4af118712fca2c679adb6caadf17c0783d0430: Status 404 returned error can't find the container with id e572b40dfafd64f4ff9f4843ce4af118712fca2c679adb6caadf17c0783d0430 Jul 08 09:13:58.824012 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.823988 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cc9fx"] Jul 08 09:13:58.826995 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.826977 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:58.829401 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.829387 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 09:13:58.836980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.836959 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-x9c7p"] Jul 08 09:13:58.838945 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:13:58.838894 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8503c860_2306_4b4c_9b33_f0c86e69d98f.slice/crio-8dddd4b15afb0332b5cb8828a217ab57cc81b5a2d91e78e1a3f435a93401bda8 WatchSource:0}: Error finding container 8dddd4b15afb0332b5cb8828a217ab57cc81b5a2d91e78e1a3f435a93401bda8: Status 404 returned error can't find the container with id 8dddd4b15afb0332b5cb8828a217ab57cc81b5a2d91e78e1a3f435a93401bda8 Jul 08 09:13:58.847021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.847001 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cc9fx"] Jul 08 09:13:58.848383 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.848315 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:58.901844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.901812 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19000ff5-718e-4a83-a56d-e01d22236c84-tls-certs\") pod \"model-serving-api-54d89bd5fc-cc9fx\" (UID: \"19000ff5-718e-4a83-a56d-e01d22236c84\") " pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:58.902221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.901854 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62brg\" (UniqueName: \"kubernetes.io/projected/19000ff5-718e-4a83-a56d-e01d22236c84-kube-api-access-62brg\") pod \"model-serving-api-54d89bd5fc-cc9fx\" (UID: \"19000ff5-718e-4a83-a56d-e01d22236c84\") " pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:58.998265 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:58.998170 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:13:59.003355 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.003320 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19000ff5-718e-4a83-a56d-e01d22236c84-tls-certs\") pod \"model-serving-api-54d89bd5fc-cc9fx\" (UID: \"19000ff5-718e-4a83-a56d-e01d22236c84\") " pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:59.003508 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.003385 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62brg\" (UniqueName: \"kubernetes.io/projected/19000ff5-718e-4a83-a56d-e01d22236c84-kube-api-access-62brg\") pod \"model-serving-api-54d89bd5fc-cc9fx\" (UID: \"19000ff5-718e-4a83-a56d-e01d22236c84\") " pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:59.005678 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.005659 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19000ff5-718e-4a83-a56d-e01d22236c84-tls-certs\") pod \"model-serving-api-54d89bd5fc-cc9fx\" (UID: \"19000ff5-718e-4a83-a56d-e01d22236c84\") " pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:59.021969 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.021944 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62brg\" (UniqueName: \"kubernetes.io/projected/19000ff5-718e-4a83-a56d-e01d22236c84-kube-api-access-62brg\") pod \"model-serving-api-54d89bd5fc-cc9fx\" (UID: \"19000ff5-718e-4a83-a56d-e01d22236c84\") " pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:59.044643 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.044608 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" event={"ID":"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823","Type":"ContainerStarted","Data":"e572b40dfafd64f4ff9f4843ce4af118712fca2c679adb6caadf17c0783d0430"} Jul 08 09:13:59.045691 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.045664 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" event={"ID":"8503c860-2306-4b4c-9b33-f0c86e69d98f","Type":"ContainerStarted","Data":"8dddd4b15afb0332b5cb8828a217ab57cc81b5a2d91e78e1a3f435a93401bda8"} Jul 08 09:13:59.138262 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.138223 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:13:59.320572 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:13:59.320501 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cc9fx"] Jul 08 09:13:59.323066 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:13:59.323042 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19000ff5_718e_4a83_a56d_e01d22236c84.slice/crio-218aaf6dfc21eeea6931a31d7fee37b5cb78352783cf13de614ea6347b24ee6a WatchSource:0}: Error finding container 218aaf6dfc21eeea6931a31d7fee37b5cb78352783cf13de614ea6347b24ee6a: Status 404 returned error can't find the container with id 218aaf6dfc21eeea6931a31d7fee37b5cb78352783cf13de614ea6347b24ee6a Jul 08 09:14:00.056153 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.056113 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" event={"ID":"19000ff5-718e-4a83-a56d-e01d22236c84","Type":"ContainerStarted","Data":"218aaf6dfc21eeea6931a31d7fee37b5cb78352783cf13de614ea6347b24ee6a"} Jul 08 09:14:00.118482 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.118447 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4"] Jul 08 09:14:00.121440 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.121417 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.124338 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.124260 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 09:14:00.124521 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.124502 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w4rnw\"" Jul 08 09:14:00.124700 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.124682 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 09:14:00.124975 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.124956 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 09:14:00.127665 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.127646 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 09:14:00.140582 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.140554 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:00.142886 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.142861 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4"] Jul 08 09:14:00.216114 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216074 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htll7\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-kube-api-access-htll7\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.216298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216141 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dcb555-b491-4231-8e6d-b25df5273f33-local-certs\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.216298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.216298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216234 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-istio-token\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.216298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216256 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.216298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216280 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.216567 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.216315 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-cacerts\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.218511 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.218481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:00.317674 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317587 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htll7\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-kube-api-access-htll7\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.317674 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317658 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dcb555-b491-4231-8e6d-b25df5273f33-local-certs\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.318057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317731 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.318057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-istio-token\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.318057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317791 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.318057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317871 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.318057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.317930 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-cacerts\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.318714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.318689 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.321140 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.321080 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dcb555-b491-4231-8e6d-b25df5273f33-local-certs\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.321140 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.321087 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-cacerts\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.321791 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.321661 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.321791 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.321749 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.332794 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.332768 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htll7\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-kube-api-access-htll7\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.336298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.336256 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-istio-token\") pod \"istiod-openshift-gateway-7574459656-qzlh4\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.437360 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.437324 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:00.623858 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:00.623824 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4"] Jul 08 09:14:00.625995 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:14:00.625971 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57dcb555_b491_4231_8e6d_b25df5273f33.slice/crio-def4b3e6d1c5c2ecdd6dfe9518008d98d259b25c2bf8bbcfcf6f04dc6a715822 WatchSource:0}: Error finding container def4b3e6d1c5c2ecdd6dfe9518008d98d259b25c2bf8bbcfcf6f04dc6a715822: Status 404 returned error can't find the container with id def4b3e6d1c5c2ecdd6dfe9518008d98d259b25c2bf8bbcfcf6f04dc6a715822 Jul 08 09:14:01.064000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:01.063885 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" event={"ID":"57dcb555-b491-4231-8e6d-b25df5273f33","Type":"ContainerStarted","Data":"def4b3e6d1c5c2ecdd6dfe9518008d98d259b25c2bf8bbcfcf6f04dc6a715822"} Jul 08 09:14:01.636559 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:01.636514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:04.333038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:04.333016 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 09:14:04.385267 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:04.385200 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:04.385483 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:04.385325 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:05.068721 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.068676 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:05.097302 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.097259 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" event={"ID":"57dcb555-b491-4231-8e6d-b25df5273f33","Type":"ContainerStarted","Data":"7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b"} Jul 08 09:14:05.097514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.097333 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:05.099324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.099298 2572 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qzlh4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 09:14:05.099767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.099745 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" podUID="57dcb555-b491-4231-8e6d-b25df5273f33" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 09:14:05.099920 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.099321 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" event={"ID":"8503c860-2306-4b4c-9b33-f0c86e69d98f","Type":"ContainerStarted","Data":"5fa2ac809fdaa980b7a652f0c4f28a6e82eb3d929a801c52158379cf3c50a6f7"} Jul 08 09:14:05.099920 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.099827 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:14:05.101166 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.101141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" event={"ID":"d12c84a8-3768-4dbe-8fa3-1b58d3c9f823","Type":"ContainerStarted","Data":"ceba6a7f5ecb94ebfe614d454f415bd2f022546b8b0e3b85c945a5b34bdfe18b"} Jul 08 09:14:05.102951 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.102924 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" event={"ID":"19000ff5-718e-4a83-a56d-e01d22236c84","Type":"ContainerStarted","Data":"8a83376f315fc06a5531214e053cb7066a6128b5cd87a404d17a5a57f63a6b82"} Jul 08 09:14:05.103036 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.102990 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:14:05.131558 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.131344 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" podStartSLOduration=1.374408019 podStartE2EDuration="5.131323137s" totalImagesPullingTime="3.756915118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:05.129397285 +0000 UTC m=+423.484408859" watchObservedRunningTime="2026-07-08 09:14:05.131323137 +0000 UTC m=+423.486334699" Jul 08 09:14:05.154261 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.154210 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5458fdb769-sv77r" podStartSLOduration=1.623806954 podStartE2EDuration="7.154192616s" totalImagesPullingTime="5.530385662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:05.153164788 +0000 UTC m=+423.508176371" watchObservedRunningTime="2026-07-08 09:14:05.154192616 +0000 UTC m=+423.509204179" Jul 08 09:14:05.163612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.163575 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:05.182102 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.182050 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" podStartSLOduration=2.699682665 podStartE2EDuration="8.182029999s" totalImagesPullingTime="5.482347334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:05.179312821 +0000 UTC m=+423.534324404" watchObservedRunningTime="2026-07-08 09:14:05.182029999 +0000 UTC m=+423.537041562" Jul 08 09:14:05.209686 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.209629 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" podStartSLOduration=2.212287865 podStartE2EDuration="7.209607784s" totalImagesPullingTime="4.997319919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:05.206714082 +0000 UTC m=+423.561725645" watchObservedRunningTime="2026-07-08 09:14:05.209607784 +0000 UTC m=+423.564619347" Jul 08 09:14:05.981863 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.981829 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5"] Jul 08 09:14:05.985679 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.985659 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:05.988793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:05.988765 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-n95bc\"" Jul 08 09:14:06.026242 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.026203 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:06.053922 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.053862 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5"] Jul 08 09:14:06.078512 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078479 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078537 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4514cb1-0237-4496-a79f-a194e578c664-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078618 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078645 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2xcl\" (UniqueName: \"kubernetes.io/projected/e4514cb1-0237-4496-a79f-a194e578c664-kube-api-access-m2xcl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078675 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078939 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078714 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4514cb1-0237-4496-a79f-a194e578c664-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078939 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4514cb1-0237-4496-a79f-a194e578c664-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078939 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078786 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.078939 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.078818 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.110739 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.110711 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:06.153320 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.153282 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:06.180021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.179992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4514cb1-0237-4496-a79f-a194e578c664-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.180199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180074 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.180199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2xcl\" (UniqueName: \"kubernetes.io/projected/e4514cb1-0237-4496-a79f-a194e578c664-kube-api-access-m2xcl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180596 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4514cb1-0237-4496-a79f-a194e578c664-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180629 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4514cb1-0237-4496-a79f-a194e578c664-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180762 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180917 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.180944 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.181245 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.181299 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.181563 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4514cb1-0237-4496-a79f-a194e578c664-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.181770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.181719 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.183110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.183076 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4514cb1-0237-4496-a79f-a194e578c664-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.183378 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.183358 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4514cb1-0237-4496-a79f-a194e578c664-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.188185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.188160 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2xcl\" (UniqueName: \"kubernetes.io/projected/e4514cb1-0237-4496-a79f-a194e578c664-kube-api-access-m2xcl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.191466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.191430 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4514cb1-0237-4496-a79f-a194e578c664-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4nw4h5\" (UID: \"e4514cb1-0237-4496-a79f-a194e578c664\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.248141 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.248039 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:06.297278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.297247 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:06.471720 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:06.471693 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5"] Jul 08 09:14:06.473296 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:14:06.473269 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4514cb1_0237_4496_a79f_a194e578c664.slice/crio-85a17cbaf889dbd16b5ad5e1965f9529ee127e5ad6138930922ea0311b8bef81 WatchSource:0}: Error finding container 85a17cbaf889dbd16b5ad5e1965f9529ee127e5ad6138930922ea0311b8bef81: Status 404 returned error can't find the container with id 85a17cbaf889dbd16b5ad5e1965f9529ee127e5ad6138930922ea0311b8bef81 Jul 08 09:14:07.117359 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:07.117324 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" event={"ID":"e4514cb1-0237-4496-a79f-a194e578c664","Type":"ContainerStarted","Data":"85a17cbaf889dbd16b5ad5e1965f9529ee127e5ad6138930922ea0311b8bef81"} Jul 08 09:14:08.179932 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.179864 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:08.311770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.311726 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn"] Jul 08 09:14:08.321606 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.321575 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:08.325522 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.325498 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fb855\"" Jul 08 09:14:08.325985 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.325961 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 09:14:08.356729 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.356701 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn"] Jul 08 09:14:08.386154 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.386116 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:08.399517 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.399482 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp"] Jul 08 09:14:08.403635 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.403613 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:08.404082 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.404046 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28d7ead6-d196-460d-9fe8-800d533065ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:08.404256 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.404096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkbkm\" (UniqueName: \"kubernetes.io/projected/28d7ead6-d196-460d-9fe8-800d533065ac-kube-api-access-vkbkm\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:08.409157 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.409135 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wqftm\"" Jul 08 09:14:08.417308 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.417284 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 09:14:08.432455 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.432381 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp"] Jul 08 09:14:08.505554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.505513 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3672feac-433d-4794-b5b2-8e19fb40be55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:08.505740 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.505649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28d7ead6-d196-460d-9fe8-800d533065ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:08.505740 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.505690 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6nn4\" (UniqueName: \"kubernetes.io/projected/3672feac-433d-4794-b5b2-8e19fb40be55-kube-api-access-r6nn4\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:08.505740 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.505726 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbkm\" (UniqueName: \"kubernetes.io/projected/28d7ead6-d196-460d-9fe8-800d533065ac-kube-api-access-vkbkm\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:08.505933 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:14:08.505798 2572 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 08 09:14:08.505933 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:14:08.505892 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28d7ead6-d196-460d-9fe8-800d533065ac-cert podName:28d7ead6-d196-460d-9fe8-800d533065ac nodeName:}" failed. No retries permitted until 2026-07-08 09:14:09.005868632 +0000 UTC m=+427.360880194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/28d7ead6-d196-460d-9fe8-800d533065ac-cert") pod "kserve-controller-manager-d6f4b65b8-v6qbn" (UID: "28d7ead6-d196-460d-9fe8-800d533065ac") : secret "kserve-webhook-server-cert" not found Jul 08 09:14:08.530324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.530297 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkbkm\" (UniqueName: \"kubernetes.io/projected/28d7ead6-d196-460d-9fe8-800d533065ac-kube-api-access-vkbkm\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:08.607187 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.607153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3672feac-433d-4794-b5b2-8e19fb40be55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:08.607373 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.607269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6nn4\" (UniqueName: \"kubernetes.io/projected/3672feac-433d-4794-b5b2-8e19fb40be55-kube-api-access-r6nn4\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:08.607373 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:14:08.607337 2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 08 09:14:08.607505 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:14:08.607434 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3672feac-433d-4794-b5b2-8e19fb40be55-cert podName:3672feac-433d-4794-b5b2-8e19fb40be55 nodeName:}" failed. No retries permitted until 2026-07-08 09:14:09.107412226 +0000 UTC m=+427.462423770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3672feac-433d-4794-b5b2-8e19fb40be55-cert") pod "llmisvc-controller-manager-777c6bd7d5-bc2zp" (UID: "3672feac-433d-4794-b5b2-8e19fb40be55") : secret "llmisvc-webhook-server-cert" not found Jul 08 09:14:08.627475 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.627447 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6nn4\" (UniqueName: \"kubernetes.io/projected/3672feac-433d-4794-b5b2-8e19fb40be55-kube-api-access-r6nn4\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:08.708189 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.708080 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:08.844855 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:08.844803 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:09.005923 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.005815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:09.011661 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.011627 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28d7ead6-d196-460d-9fe8-800d533065ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:09.014672 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.014645 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28d7ead6-d196-460d-9fe8-800d533065ac-cert\") pod \"kserve-controller-manager-d6f4b65b8-v6qbn\" (UID: \"28d7ead6-d196-460d-9fe8-800d533065ac\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:09.112325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.112300 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3672feac-433d-4794-b5b2-8e19fb40be55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:09.114874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.114828 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3672feac-433d-4794-b5b2-8e19fb40be55-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bc2zp\" (UID: \"3672feac-433d-4794-b5b2-8e19fb40be55\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:09.154851 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.154820 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:09.154957 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.154910 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:09.154957 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.154945 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:14:09.233825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.233793 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:09.314509 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.314263 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:09.416782 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.416704 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn"] Jul 08 09:14:09.504093 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:09.504070 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp"] Jul 08 09:14:09.506098 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:14:09.506073 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3672feac_433d_4794_b5b2_8e19fb40be55.slice/crio-1d8de6879a2cf9c8d4ad4c161b3871d9dd705f3f768d4f71977c26edc5ede934 WatchSource:0}: Error finding container 1d8de6879a2cf9c8d4ad4c161b3871d9dd705f3f768d4f71977c26edc5ede934: Status 404 returned error can't find the container with id 1d8de6879a2cf9c8d4ad4c161b3871d9dd705f3f768d4f71977c26edc5ede934 Jul 08 09:14:10.041158 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.041118 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4"] Jul 08 09:14:10.041331 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.041308 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" podUID="57dcb555-b491-4231-8e6d-b25df5273f33" containerName="discovery" containerID="cri-o://7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b" gracePeriod=30 Jul 08 09:14:10.138565 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.138529 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" event={"ID":"e4514cb1-0237-4496-a79f-a194e578c664","Type":"ContainerStarted","Data":"0b495aecb0b148fbb25feeb3fd95e0f9ec4f15d06e374d88e4c626cd03153a49"} Jul 08 09:14:10.139787 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.139758 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" event={"ID":"28d7ead6-d196-460d-9fe8-800d533065ac","Type":"ContainerStarted","Data":"caef14a8beccef8677e42eafe7494f9df832dba2334baf3914e8bec2d451e213"} Jul 08 09:14:10.141209 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.141179 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" event={"ID":"3672feac-433d-4794-b5b2-8e19fb40be55","Type":"ContainerStarted","Data":"1d8de6879a2cf9c8d4ad4c161b3871d9dd705f3f768d4f71977c26edc5ede934"} Jul 08 09:14:10.166427 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.166376 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" podStartSLOduration=2.487233527 podStartE2EDuration="5.16635828s" totalImagesPullingTime="2.679124753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:10.163530307 +0000 UTC m=+428.518541869" watchObservedRunningTime="2026-07-08 09:14:10.16635828 +0000 UTC m=+428.521369841" Jul 08 09:14:10.298184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.298101 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:10.302681 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.302658 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:10.340204 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.340179 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:10.424970 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.424939 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htll7\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-kube-api-access-htll7\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425139 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.424978 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-kubeconfig\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425139 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.425006 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-dns-cert\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425139 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.425039 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dcb555-b491-4231-8e6d-b25df5273f33-local-certs\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425139 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.425096 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-ca-configmap\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.425141 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-cacerts\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.425203 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-istio-token\") pod \"57dcb555-b491-4231-8e6d-b25df5273f33\" (UID: \"57dcb555-b491-4231-8e6d-b25df5273f33\") " Jul 08 09:14:10.425685 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.425564 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:14:10.428127 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.428094 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-cacerts" (OuterVolumeSpecName: "cacerts") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:14:10.428225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.428211 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-istio-token" (OuterVolumeSpecName: "istio-token") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:14:10.428300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.428264 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:14:10.428358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.428309 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:14:10.428409 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.428370 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57dcb555-b491-4231-8e6d-b25df5273f33-local-certs" (OuterVolumeSpecName: "local-certs") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:14:10.428550 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.428516 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-kube-api-access-htll7" (OuterVolumeSpecName: "kube-api-access-htll7") pod "57dcb555-b491-4231-8e6d-b25df5273f33" (UID: "57dcb555-b491-4231-8e6d-b25df5273f33"). InnerVolumeSpecName "kube-api-access-htll7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:14:10.526285 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526254 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-ca-configmap\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:10.526285 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526286 2572 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-cacerts\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:10.526466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526297 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-istio-token\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:10.526466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526305 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htll7\" (UniqueName: \"kubernetes.io/projected/57dcb555-b491-4231-8e6d-b25df5273f33-kube-api-access-htll7\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:10.526466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526313 2572 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-kubeconfig\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:10.526466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526321 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/57dcb555-b491-4231-8e6d-b25df5273f33-istio-csr-dns-cert\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:10.526466 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:10.526329 2572 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/57dcb555-b491-4231-8e6d-b25df5273f33-local-certs\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:14:11.146828 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.146793 2572 generic.go:356] "Generic (PLEG): container finished" podID="57dcb555-b491-4231-8e6d-b25df5273f33" containerID="7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b" exitCode=0 Jul 08 09:14:11.147014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.146835 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" Jul 08 09:14:11.147014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.146883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" event={"ID":"57dcb555-b491-4231-8e6d-b25df5273f33","Type":"ContainerDied","Data":"7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b"} Jul 08 09:14:11.147014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.146948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4" event={"ID":"57dcb555-b491-4231-8e6d-b25df5273f33","Type":"ContainerDied","Data":"def4b3e6d1c5c2ecdd6dfe9518008d98d259b25c2bf8bbcfcf6f04dc6a715822"} Jul 08 09:14:11.147014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.146977 2572 scope.go:117] "RemoveContainer" containerID="7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b" Jul 08 09:14:11.147390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.147367 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:11.148349 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.148327 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4nw4h5" Jul 08 09:14:11.155813 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.155798 2572 scope.go:117] "RemoveContainer" containerID="7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b" Jul 08 09:14:11.156066 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:14:11.156048 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b\": container with ID starting with 7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b not found: ID does not exist" containerID="7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b" Jul 08 09:14:11.156129 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.156075 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b"} err="failed to get container status \"7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b\": rpc error: code = NotFound desc = could not find container \"7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b\": container with ID starting with 7fa1b16f6350ef3d0690370b8d635cf1dec3517442febab2747a60ff818c8a9b not found: ID does not exist" Jul 08 09:14:11.190759 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.190728 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4"] Jul 08 09:14:11.199468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:11.199446 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qzlh4"] Jul 08 09:14:12.175547 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:12.175512 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57dcb555-b491-4231-8e6d-b25df5273f33" path="/var/lib/kubelet/pods/57dcb555-b491-4231-8e6d-b25df5273f33/volumes" Jul 08 09:14:14.165324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:14.165275 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" event={"ID":"3672feac-433d-4794-b5b2-8e19fb40be55","Type":"ContainerStarted","Data":"fd89ae2cc064566f90ba2f3efc0cec9396a17f4d7b3dca74ebae46611f6d292d"} Jul 08 09:14:14.165850 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:14.165579 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:14.166968 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:14.166939 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" event={"ID":"28d7ead6-d196-460d-9fe8-800d533065ac","Type":"ContainerStarted","Data":"a4c4e78041176b22453307575293c3cf0b322ae4633f8fd8bd10c56e7fc2a92d"} Jul 08 09:14:14.167087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:14.167019 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:14.185437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:14.185391 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" podStartSLOduration=2.189069308 podStartE2EDuration="6.185376083s" totalImagesPullingTime="3.996306775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:14.184560375 +0000 UTC m=+432.539571938" watchObservedRunningTime="2026-07-08 09:14:14.185376083 +0000 UTC m=+432.540387645" Jul 08 09:14:16.113860 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:16.113833 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cc9fx" Jul 08 09:14:16.132202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:16.132147 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" podStartSLOduration=4.055902305 podStartE2EDuration="8.132132085s" totalImagesPullingTime="4.07622978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:14:14.209581465 +0000 UTC m=+432.564593048" watchObservedRunningTime="2026-07-08 09:14:16.132132085 +0000 UTC m=+434.487143647" Jul 08 09:14:17.120438 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:17.120400 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-x9c7p" Jul 08 09:14:45.174388 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:45.174356 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bc2zp" Jul 08 09:14:45.177281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:45.177256 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-v6qbn" Jul 08 09:14:58.432672 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.432640 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:58.531807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.531773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:58.569994 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.569965 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2gtph"] Jul 08 09:14:58.570382 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.570367 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57dcb555-b491-4231-8e6d-b25df5273f33" containerName="discovery" Jul 08 09:14:58.570430 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.570383 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57dcb555-b491-4231-8e6d-b25df5273f33" containerName="discovery" Jul 08 09:14:58.570468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.570452 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="57dcb555-b491-4231-8e6d-b25df5273f33" containerName="discovery" Jul 08 09:14:58.572918 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.572879 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:14:58.575916 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.575880 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 09:14:58.576591 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.576568 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9k8rx\"" Jul 08 09:14:58.576710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.576602 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 09:14:58.582334 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.582147 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2gtph"] Jul 08 09:14:58.630203 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.630170 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:58.667541 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.667515 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzj9k\" (UniqueName: \"kubernetes.io/projected/9154c5b4-5876-4b36-921a-ae01ddfedc65-kube-api-access-xzj9k\") pod \"kuadrant-operator-catalog-2gtph\" (UID: \"9154c5b4-5876-4b36-921a-ae01ddfedc65\") " pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:14:58.769020 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.768929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzj9k\" (UniqueName: \"kubernetes.io/projected/9154c5b4-5876-4b36-921a-ae01ddfedc65-kube-api-access-xzj9k\") pod \"kuadrant-operator-catalog-2gtph\" (UID: \"9154c5b4-5876-4b36-921a-ae01ddfedc65\") " pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:14:58.779363 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.779335 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzj9k\" (UniqueName: \"kubernetes.io/projected/9154c5b4-5876-4b36-921a-ae01ddfedc65-kube-api-access-xzj9k\") pod \"kuadrant-operator-catalog-2gtph\" (UID: \"9154c5b4-5876-4b36-921a-ae01ddfedc65\") " pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:14:58.885382 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.885352 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:14:58.922035 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:58.922007 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2gtph"] Jul 08 09:14:59.017506 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.017410 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2gtph"] Jul 08 09:14:59.020400 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:14:59.020331 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9154c5b4_5876_4b36_921a_ae01ddfedc65.slice/crio-d16db227c6fa797c99f55f835ce4033a2251fe63dce917af399efbbf130907ca WatchSource:0}: Error finding container d16db227c6fa797c99f55f835ce4033a2251fe63dce917af399efbbf130907ca: Status 404 returned error can't find the container with id d16db227c6fa797c99f55f835ce4033a2251fe63dce917af399efbbf130907ca Jul 08 09:14:59.132050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.132006 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cl526"] Jul 08 09:14:59.135469 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.135453 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:14:59.143302 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.143278 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cl526"] Jul 08 09:14:59.274292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.274178 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4lwk\" (UniqueName: \"kubernetes.io/projected/32e2ca96-83dc-4272-8743-fb7531f0ef09-kube-api-access-j4lwk\") pod \"kuadrant-operator-catalog-cl526\" (UID: \"32e2ca96-83dc-4272-8743-fb7531f0ef09\") " pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:14:59.367035 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.367002 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:59.372617 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.372586 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" event={"ID":"9154c5b4-5876-4b36-921a-ae01ddfedc65","Type":"ContainerStarted","Data":"d16db227c6fa797c99f55f835ce4033a2251fe63dce917af399efbbf130907ca"} Jul 08 09:14:59.375152 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.375130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4lwk\" (UniqueName: \"kubernetes.io/projected/32e2ca96-83dc-4272-8743-fb7531f0ef09-kube-api-access-j4lwk\") pod \"kuadrant-operator-catalog-cl526\" (UID: \"32e2ca96-83dc-4272-8743-fb7531f0ef09\") " pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:14:59.384093 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.384068 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4lwk\" (UniqueName: \"kubernetes.io/projected/32e2ca96-83dc-4272-8743-fb7531f0ef09-kube-api-access-j4lwk\") pod \"kuadrant-operator-catalog-cl526\" (UID: \"32e2ca96-83dc-4272-8743-fb7531f0ef09\") " pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:14:59.446185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.446157 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:14:59.576381 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.576328 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:14:59.676867 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:14:59.676792 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cl526"] Jul 08 09:14:59.749413 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:14:59.749380 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e2ca96_83dc_4272_8743_fb7531f0ef09.slice/crio-bac0285df1e9ccbb22e94beb779ce3366578b9b803fd78d62439f6844a4673bb WatchSource:0}: Error finding container bac0285df1e9ccbb22e94beb779ce3366578b9b803fd78d62439f6844a4673bb: Status 404 returned error can't find the container with id bac0285df1e9ccbb22e94beb779ce3366578b9b803fd78d62439f6844a4673bb Jul 08 09:15:00.378479 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:00.378444 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cl526" event={"ID":"32e2ca96-83dc-4272-8743-fb7531f0ef09","Type":"ContainerStarted","Data":"bac0285df1e9ccbb22e94beb779ce3366578b9b803fd78d62439f6844a4673bb"} Jul 08 09:15:01.384358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.384321 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" event={"ID":"9154c5b4-5876-4b36-921a-ae01ddfedc65","Type":"ContainerStarted","Data":"9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320"} Jul 08 09:15:01.384825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.384361 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" podUID="9154c5b4-5876-4b36-921a-ae01ddfedc65" containerName="registry-server" containerID="cri-o://9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320" gracePeriod=2 Jul 08 09:15:01.385728 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.385706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cl526" event={"ID":"32e2ca96-83dc-4272-8743-fb7531f0ef09","Type":"ContainerStarted","Data":"3846554146bdd2848855c6053aba5259dee7b06f73182cd45e89e5dbd6b26c01"} Jul 08 09:15:01.404804 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.404760 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" podStartSLOduration=1.382561549 podStartE2EDuration="3.404744775s" totalImagesPullingTime="2.022183226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:15:01.40258375 +0000 UTC m=+479.757595313" watchObservedRunningTime="2026-07-08 09:15:01.404744775 +0000 UTC m=+479.759756337" Jul 08 09:15:01.422891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.422839 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cl526" podStartSLOduration=1.133828701 podStartE2EDuration="2.422822869s" totalImagesPullingTime="1.288994168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:15:01.419603987 +0000 UTC m=+479.774615550" watchObservedRunningTime="2026-07-08 09:15:01.422822869 +0000 UTC m=+479.777834431" Jul 08 09:15:01.629166 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.629140 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:15:01.798222 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.798135 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzj9k\" (UniqueName: \"kubernetes.io/projected/9154c5b4-5876-4b36-921a-ae01ddfedc65-kube-api-access-xzj9k\") pod \"9154c5b4-5876-4b36-921a-ae01ddfedc65\" (UID: \"9154c5b4-5876-4b36-921a-ae01ddfedc65\") " Jul 08 09:15:01.800310 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.800285 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9154c5b4-5876-4b36-921a-ae01ddfedc65-kube-api-access-xzj9k" (OuterVolumeSpecName: "kube-api-access-xzj9k") pod "9154c5b4-5876-4b36-921a-ae01ddfedc65" (UID: "9154c5b4-5876-4b36-921a-ae01ddfedc65"). InnerVolumeSpecName "kube-api-access-xzj9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:15:01.899093 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:01.899060 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzj9k\" (UniqueName: \"kubernetes.io/projected/9154c5b4-5876-4b36-921a-ae01ddfedc65-kube-api-access-xzj9k\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:15:02.397727 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.397689 2572 generic.go:356] "Generic (PLEG): container finished" podID="9154c5b4-5876-4b36-921a-ae01ddfedc65" containerID="9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320" exitCode=0 Jul 08 09:15:02.398225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.397754 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" Jul 08 09:15:02.398225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.397776 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" event={"ID":"9154c5b4-5876-4b36-921a-ae01ddfedc65","Type":"ContainerDied","Data":"9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320"} Jul 08 09:15:02.398225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.397846 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2gtph" event={"ID":"9154c5b4-5876-4b36-921a-ae01ddfedc65","Type":"ContainerDied","Data":"d16db227c6fa797c99f55f835ce4033a2251fe63dce917af399efbbf130907ca"} Jul 08 09:15:02.398225 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.397868 2572 scope.go:117] "RemoveContainer" containerID="9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320" Jul 08 09:15:02.407662 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.407643 2572 scope.go:117] "RemoveContainer" containerID="9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320" Jul 08 09:15:02.407940 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:15:02.407919 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320\": container with ID starting with 9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320 not found: ID does not exist" containerID="9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320" Jul 08 09:15:02.408013 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.407951 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320"} err="failed to get container status \"9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320\": rpc error: code = NotFound desc = could not find container \"9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320\": container with ID starting with 9694f2177bf27826730f4fd51c577f73c0998a7eb74e07b83382a030d7b80320 not found: ID does not exist" Jul 08 09:15:02.414869 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.414845 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2gtph"] Jul 08 09:15:02.419052 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:02.419031 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2gtph"] Jul 08 09:15:04.175494 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:04.175462 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9154c5b4-5876-4b36-921a-ae01ddfedc65" path="/var/lib/kubelet/pods/9154c5b4-5876-4b36-921a-ae01ddfedc65/volumes" Jul 08 09:15:09.446929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:09.446884 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:15:09.447362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:09.446954 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:15:09.469040 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:09.469017 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:15:10.457382 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:10.457356 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-cl526" Jul 08 09:15:22.174656 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:22.174618 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:24.513150 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.513117 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw"] Jul 08 09:15:24.513572 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.513556 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9154c5b4-5876-4b36-921a-ae01ddfedc65" containerName="registry-server" Jul 08 09:15:24.513621 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.513574 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9154c5b4-5876-4b36-921a-ae01ddfedc65" containerName="registry-server" Jul 08 09:15:24.513656 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.513633 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9154c5b4-5876-4b36-921a-ae01ddfedc65" containerName="registry-server" Jul 08 09:15:24.515670 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.515654 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:24.520406 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.520378 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 09:15:24.520655 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.520641 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7glnl\"" Jul 08 09:15:24.537064 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.537042 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw"] Jul 08 09:15:24.596586 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.596556 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-628b6\" (UniqueName: \"kubernetes.io/projected/0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a-kube-api-access-628b6\") pod \"dns-operator-controller-manager-864dd78c4c-qnrvw\" (UID: \"0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:24.697252 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.697220 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-628b6\" (UniqueName: \"kubernetes.io/projected/0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a-kube-api-access-628b6\") pod \"dns-operator-controller-manager-864dd78c4c-qnrvw\" (UID: \"0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:24.759560 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.759533 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-628b6\" (UniqueName: \"kubernetes.io/projected/0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a-kube-api-access-628b6\") pod \"dns-operator-controller-manager-864dd78c4c-qnrvw\" (UID: \"0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:24.825464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:24.825382 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:25.011087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:25.011056 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw"] Jul 08 09:15:25.011732 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:15:25.011705 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0148feb3_6cbb_4a98_b5ff_4d063a3e1e4a.slice/crio-eccc85a01ad7b2331896aa5be6651d3e2b91f2512cc7f45b604505dc3f439dc6 WatchSource:0}: Error finding container eccc85a01ad7b2331896aa5be6651d3e2b91f2512cc7f45b604505dc3f439dc6: Status 404 returned error can't find the container with id eccc85a01ad7b2331896aa5be6651d3e2b91f2512cc7f45b604505dc3f439dc6 Jul 08 09:15:25.485694 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:25.485663 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:25.507001 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:25.506967 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" event={"ID":"0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a","Type":"ContainerStarted","Data":"eccc85a01ad7b2331896aa5be6651d3e2b91f2512cc7f45b604505dc3f439dc6"} Jul 08 09:15:26.194657 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:26.194548 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:26.999237 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:26.999190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:27.199913 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:27.199828 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:27.485420 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:27.485384 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:27.518549 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:27.518515 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" event={"ID":"0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a","Type":"ContainerStarted","Data":"c1bee1669b2d4ccd51605a2c8446bc0f5df195a29145ec91981b5ba0bb09571d"} Jul 08 09:15:27.518719 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:27.518644 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:27.562873 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:27.562812 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" podStartSLOduration=1.236128809 podStartE2EDuration="3.562791642s" totalImagesPullingTime="2.326662833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:15:27.560981985 +0000 UTC m=+505.915993550" watchObservedRunningTime="2026-07-08 09:15:27.562791642 +0000 UTC m=+505.917803204" Jul 08 09:15:27.616860 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:27.616833 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:29.095272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.095240 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm"] Jul 08 09:15:29.097751 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.097736 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:29.100521 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.100489 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rbx2q\"" Jul 08 09:15:29.114864 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.114841 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm"] Jul 08 09:15:29.142171 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.142145 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9w9\" (UniqueName: \"kubernetes.io/projected/f3d6f712-e668-4989-8f08-d1ad02d937ca-kube-api-access-zj9w9\") pod \"authorino-operator-75cc8d5cb7-zbqfm\" (UID: \"f3d6f712-e668-4989-8f08-d1ad02d937ca\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:29.243113 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.243084 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9w9\" (UniqueName: \"kubernetes.io/projected/f3d6f712-e668-4989-8f08-d1ad02d937ca-kube-api-access-zj9w9\") pod \"authorino-operator-75cc8d5cb7-zbqfm\" (UID: \"f3d6f712-e668-4989-8f08-d1ad02d937ca\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:29.257375 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.257350 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9w9\" (UniqueName: \"kubernetes.io/projected/f3d6f712-e668-4989-8f08-d1ad02d937ca-kube-api-access-zj9w9\") pod \"authorino-operator-75cc8d5cb7-zbqfm\" (UID: \"f3d6f712-e668-4989-8f08-d1ad02d937ca\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:29.407497 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.407468 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:29.561724 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:29.561703 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm"] Jul 08 09:15:29.563848 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:15:29.563822 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d6f712_e668_4989_8f08_d1ad02d937ca.slice/crio-280f113690855beebf8c1290756266bed4b639a9e9254b9fc7cc0bb397759753 WatchSource:0}: Error finding container 280f113690855beebf8c1290756266bed4b639a9e9254b9fc7cc0bb397759753: Status 404 returned error can't find the container with id 280f113690855beebf8c1290756266bed4b639a9e9254b9fc7cc0bb397759753 Jul 08 09:15:30.170201 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:30.170166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:30.542015 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:30.541839 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" event={"ID":"f3d6f712-e668-4989-8f08-d1ad02d937ca","Type":"ContainerStarted","Data":"280f113690855beebf8c1290756266bed4b639a9e9254b9fc7cc0bb397759753"} Jul 08 09:15:32.164087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.164056 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx"] Jul 08 09:15:32.167555 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.167537 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:32.170477 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.170454 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wd8rc\"" Jul 08 09:15:32.181172 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.181148 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx"] Jul 08 09:15:32.275708 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.275608 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pbgh\" (UniqueName: \"kubernetes.io/projected/2a9f4697-fabf-482a-a5bd-345675adfad2-kube-api-access-5pbgh\") pod \"limitador-operator-controller-manager-859d7b78bd-r47dx\" (UID: \"2a9f4697-fabf-482a-a5bd-345675adfad2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:32.376765 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.376730 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pbgh\" (UniqueName: \"kubernetes.io/projected/2a9f4697-fabf-482a-a5bd-345675adfad2-kube-api-access-5pbgh\") pod \"limitador-operator-controller-manager-859d7b78bd-r47dx\" (UID: \"2a9f4697-fabf-482a-a5bd-345675adfad2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:32.390152 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.390125 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pbgh\" (UniqueName: \"kubernetes.io/projected/2a9f4697-fabf-482a-a5bd-345675adfad2-kube-api-access-5pbgh\") pod \"limitador-operator-controller-manager-859d7b78bd-r47dx\" (UID: \"2a9f4697-fabf-482a-a5bd-345675adfad2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:32.477949 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.477892 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:32.484368 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.484339 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:32.554331 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.554300 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" event={"ID":"f3d6f712-e668-4989-8f08-d1ad02d937ca","Type":"ContainerStarted","Data":"1f2d2383c4353e7f99ee2916cb7865c65231a4dbc9fcfd295040c6f59058c4b0"} Jul 08 09:15:32.554495 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.554421 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:32.579503 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.579456 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" podStartSLOduration=1.177298082 podStartE2EDuration="3.579440682s" totalImagesPullingTime="2.4021426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:15:32.577653882 +0000 UTC m=+510.932665465" watchObservedRunningTime="2026-07-08 09:15:32.579440682 +0000 UTC m=+510.934452244" Jul 08 09:15:32.638056 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.638035 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx"] Jul 08 09:15:32.640960 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:15:32.640927 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9f4697_fabf_482a_a5bd_345675adfad2.slice/crio-690c1835095948a515e6cca48bb4fe932a633d5bfd701ad40414d0592ec49af2 WatchSource:0}: Error finding container 690c1835095948a515e6cca48bb4fe932a633d5bfd701ad40414d0592ec49af2: Status 404 returned error can't find the container with id 690c1835095948a515e6cca48bb4fe932a633d5bfd701ad40414d0592ec49af2 Jul 08 09:15:32.805296 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:32.805197 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:33.562114 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:33.562064 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" event={"ID":"2a9f4697-fabf-482a-a5bd-345675adfad2","Type":"ContainerStarted","Data":"690c1835095948a515e6cca48bb4fe932a633d5bfd701ad40414d0592ec49af2"} Jul 08 09:15:33.683527 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:33.683496 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:34.568604 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:34.568571 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" event={"ID":"2a9f4697-fabf-482a-a5bd-345675adfad2","Type":"ContainerStarted","Data":"2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc"} Jul 08 09:15:34.568982 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:34.568696 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:34.591487 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:34.591402 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" podStartSLOduration=0.899450802 podStartE2EDuration="2.591387719s" totalImagesPullingTime="1.691936917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:15:34.589775126 +0000 UTC m=+512.944786700" watchObservedRunningTime="2026-07-08 09:15:34.591387719 +0000 UTC m=+512.946399281" Jul 08 09:15:35.532829 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:35.532784 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:37.246193 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.246158 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x"] Jul 08 09:15:37.250019 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.249988 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.252655 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.252638 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2tbss\"" Jul 08 09:15:37.264261 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.264239 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x"] Jul 08 09:15:37.322501 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.322469 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0d0c655-ce16-4b53-9e85-6a0341052c4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.322640 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.322617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gndlj\" (UniqueName: \"kubernetes.io/projected/e0d0c655-ce16-4b53-9e85-6a0341052c4f-kube-api-access-gndlj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.424011 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.423981 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0d0c655-ce16-4b53-9e85-6a0341052c4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.424179 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.424027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gndlj\" (UniqueName: \"kubernetes.io/projected/e0d0c655-ce16-4b53-9e85-6a0341052c4f-kube-api-access-gndlj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.424295 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.424272 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0d0c655-ce16-4b53-9e85-6a0341052c4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.436597 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.436567 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gndlj\" (UniqueName: \"kubernetes.io/projected/e0d0c655-ce16-4b53-9e85-6a0341052c4f-kube-api-access-gndlj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.560100 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.560027 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:37.660063 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.659883 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:37.722738 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.722674 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x"] Jul 08 09:15:37.725255 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:15:37.725227 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d0c655_ce16_4b53_9e85_6a0341052c4f.slice/crio-19cedb06a5a844b0f79dd2f7a25d9296060d87b1396a7c49558660c35d2695e6 WatchSource:0}: Error finding container 19cedb06a5a844b0f79dd2f7a25d9296060d87b1396a7c49558660c35d2695e6: Status 404 returned error can't find the container with id 19cedb06a5a844b0f79dd2f7a25d9296060d87b1396a7c49558660c35d2695e6 Jul 08 09:15:37.802831 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:37.802802 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:38.400644 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:38.400617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:38.496986 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:38.496949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:38.526700 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:38.526670 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qnrvw" Jul 08 09:15:38.593144 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:38.593102 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" event={"ID":"e0d0c655-ce16-4b53-9e85-6a0341052c4f","Type":"ContainerStarted","Data":"19cedb06a5a844b0f79dd2f7a25d9296060d87b1396a7c49558660c35d2695e6"} Jul 08 09:15:43.564997 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:43.564963 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbqfm" Jul 08 09:15:43.793842 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:43.793800 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:43.896491 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:43.896448 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:44.289468 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:44.289384 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:44.439952 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:44.439897 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:45.576340 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:45.576310 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:45.635469 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:45.635431 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" event={"ID":"e0d0c655-ce16-4b53-9e85-6a0341052c4f","Type":"ContainerStarted","Data":"77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd"} Jul 08 09:15:45.635634 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:45.635575 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:45.658554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:45.658505 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" podStartSLOduration=1.376021981 podStartE2EDuration="8.658487802s" totalImagesPullingTime="7.282465821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:15:45.656191247 +0000 UTC m=+524.011202808" watchObservedRunningTime="2026-07-08 09:15:45.658487802 +0000 UTC m=+524.013499366" Jul 08 09:15:47.738324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.738290 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr"] Jul 08 09:15:47.742312 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.742292 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:47.748050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.748028 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 09:15:47.748801 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.748783 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 09:15:47.749100 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.749085 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rxqb9\"" Jul 08 09:15:47.756287 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.756233 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:47.758825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.758778 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr"] Jul 08 09:15:47.863032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.863006 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:47.921052 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.921022 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bde984ea-c427-45ba-b5a7-8771de4ab293-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:47.921210 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.921064 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkpxm\" (UniqueName: \"kubernetes.io/projected/bde984ea-c427-45ba-b5a7-8771de4ab293-kube-api-access-bkpxm\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:47.921210 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.921097 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde984ea-c427-45ba-b5a7-8771de4ab293-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:47.983623 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:47.983583 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:48.022394 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.022295 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bde984ea-c427-45ba-b5a7-8771de4ab293-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.022394 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.022356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkpxm\" (UniqueName: \"kubernetes.io/projected/bde984ea-c427-45ba-b5a7-8771de4ab293-kube-api-access-bkpxm\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.022394 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.022390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde984ea-c427-45ba-b5a7-8771de4ab293-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.023104 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.023080 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bde984ea-c427-45ba-b5a7-8771de4ab293-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.024932 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.024895 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde984ea-c427-45ba-b5a7-8771de4ab293-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.039896 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.039866 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkpxm\" (UniqueName: \"kubernetes.io/projected/bde984ea-c427-45ba-b5a7-8771de4ab293-kube-api-access-bkpxm\") pod \"kuadrant-console-plugin-66b5774985-fwnpr\" (UID: \"bde984ea-c427-45ba-b5a7-8771de4ab293\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.050825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.050797 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" Jul 08 09:15:48.089465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.089431 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:48.183297 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.183275 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr"] Jul 08 09:15:48.185093 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:15:48.185069 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbde984ea_c427_45ba_b5a7_8771de4ab293.slice/crio-073ba17bf49e03104c6b6d30283708886f836b5cd3dbd1ef57dc0f0b775d1b3d WatchSource:0}: Error finding container 073ba17bf49e03104c6b6d30283708886f836b5cd3dbd1ef57dc0f0b775d1b3d: Status 404 returned error can't find the container with id 073ba17bf49e03104c6b6d30283708886f836b5cd3dbd1ef57dc0f0b775d1b3d Jul 08 09:15:48.650942 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:48.650910 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" event={"ID":"bde984ea-c427-45ba-b5a7-8771de4ab293","Type":"ContainerStarted","Data":"073ba17bf49e03104c6b6d30283708886f836b5cd3dbd1ef57dc0f0b775d1b3d"} Jul 08 09:15:50.392183 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:50.392145 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:50.510757 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:50.510718 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:51.450258 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:51.450211 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:53.867796 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:53.867703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:53.968363 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:53.968322 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:56.369055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:56.369010 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:56.643123 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:56.643033 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:57.063322 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:57.063220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:57.219423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:57.219369 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:15:58.475069 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.475028 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x"] Jul 08 09:15:58.475545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.475264 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" containerName="manager" containerID="cri-o://77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd" gracePeriod=2 Jul 08 09:15:58.483676 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.483627 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x"] Jul 08 09:15:58.494752 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.494659 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg"] Jul 08 09:15:58.495349 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.495327 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" containerName="manager" Jul 08 09:15:58.495493 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.495355 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" containerName="manager" Jul 08 09:15:58.495569 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.495505 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" containerName="manager" Jul 08 09:15:58.499626 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.499606 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.502764 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.502740 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx"] Jul 08 09:15:58.503000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.502955 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" containerName="manager" containerID="cri-o://2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc" gracePeriod=2 Jul 08 09:15:58.503484 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:15:58.503429 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:58.520922 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.520874 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg"] Jul 08 09:15:58.524207 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.523813 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rjtrg\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.524207 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.523941 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jzw\" (UniqueName: \"kubernetes.io/projected/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-kube-api-access-42jzw\") pod \"kuadrant-operator-controller-manager-55467d9f7-rjtrg\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.525041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.525016 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx"] Jul 08 09:15:58.536192 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.536119 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w"] Jul 08 09:15:58.536774 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.536655 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" containerName="manager" Jul 08 09:15:58.536774 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.536679 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" containerName="manager" Jul 08 09:15:58.536774 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.536757 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" containerName="manager" Jul 08 09:15:58.541315 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.541293 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:15:58.563257 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:15:58.563061 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:15:58.565626 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:15:58.565586 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-r47dx\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:15:58.568535 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.568512 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w"] Jul 08 09:15:58.624953 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.624892 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rjtrg\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.625141 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.624972 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jzw\" (UniqueName: \"kubernetes.io/projected/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-kube-api-access-42jzw\") pod \"kuadrant-operator-controller-manager-55467d9f7-rjtrg\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.625141 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.625024 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z99hm\" (UniqueName: \"kubernetes.io/projected/ec78e5c0-6923-4824-bfd1-60c1b2d854d5-kube-api-access-z99hm\") pod \"limitador-operator-controller-manager-859d7b78bd-47q2w\" (UID: \"ec78e5c0-6923-4824-bfd1-60c1b2d854d5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:15:58.625315 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.625291 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rjtrg\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.638245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.638215 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jzw\" (UniqueName: \"kubernetes.io/projected/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-kube-api-access-42jzw\") pod \"kuadrant-operator-controller-manager-55467d9f7-rjtrg\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.725966 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.725858 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z99hm\" (UniqueName: \"kubernetes.io/projected/ec78e5c0-6923-4824-bfd1-60c1b2d854d5-kube-api-access-z99hm\") pod \"limitador-operator-controller-manager-859d7b78bd-47q2w\" (UID: \"ec78e5c0-6923-4824-bfd1-60c1b2d854d5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:15:58.741078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.741043 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z99hm\" (UniqueName: \"kubernetes.io/projected/ec78e5c0-6923-4824-bfd1-60c1b2d854d5-kube-api-access-z99hm\") pod \"limitador-operator-controller-manager-859d7b78bd-47q2w\" (UID: \"ec78e5c0-6923-4824-bfd1-60c1b2d854d5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:15:58.898417 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.898384 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:15:58.905006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:15:58.904975 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:16:02.426604 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:02.426564 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:02.527330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:02.527296 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:03.099747 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:03.099701 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:03.196479 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:03.196426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:06.411314 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:06.411281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:06.510612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:06.510567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:10.223748 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:10.223707 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:10.290131 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:10.290092 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:11.393996 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.393971 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:11.396486 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.396445 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-r47dx\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:11.411275 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.411200 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:16:11.413436 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.413405 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:16:11.415533 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.415444 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-r47dx\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:11.436802 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.436773 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w"] Jul 08 09:16:11.438635 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:16:11.438607 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec78e5c0_6923_4824_bfd1_60c1b2d854d5.slice/crio-8002246b3426d5f71fc78c102716316f2ca153fc31d9ae0176d811f715d5f26f WatchSource:0}: Error finding container 8002246b3426d5f71fc78c102716316f2ca153fc31d9ae0176d811f715d5f26f: Status 404 returned error can't find the container with id 8002246b3426d5f71fc78c102716316f2ca153fc31d9ae0176d811f715d5f26f Jul 08 09:16:11.453115 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.453091 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gndlj\" (UniqueName: \"kubernetes.io/projected/e0d0c655-ce16-4b53-9e85-6a0341052c4f-kube-api-access-gndlj\") pod \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " Jul 08 09:16:11.453219 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.453181 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0d0c655-ce16-4b53-9e85-6a0341052c4f-extensions-socket-volume\") pod \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\" (UID: \"e0d0c655-ce16-4b53-9e85-6a0341052c4f\") " Jul 08 09:16:11.453304 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.453231 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pbgh\" (UniqueName: \"kubernetes.io/projected/2a9f4697-fabf-482a-a5bd-345675adfad2-kube-api-access-5pbgh\") pod \"2a9f4697-fabf-482a-a5bd-345675adfad2\" (UID: \"2a9f4697-fabf-482a-a5bd-345675adfad2\") " Jul 08 09:16:11.453692 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.453659 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d0c655-ce16-4b53-9e85-6a0341052c4f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e0d0c655-ce16-4b53-9e85-6a0341052c4f" (UID: "e0d0c655-ce16-4b53-9e85-6a0341052c4f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:16:11.455200 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.455177 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d0c655-ce16-4b53-9e85-6a0341052c4f-kube-api-access-gndlj" (OuterVolumeSpecName: "kube-api-access-gndlj") pod "e0d0c655-ce16-4b53-9e85-6a0341052c4f" (UID: "e0d0c655-ce16-4b53-9e85-6a0341052c4f"). InnerVolumeSpecName "kube-api-access-gndlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:16:11.455292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.455260 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9f4697-fabf-482a-a5bd-345675adfad2-kube-api-access-5pbgh" (OuterVolumeSpecName: "kube-api-access-5pbgh") pod "2a9f4697-fabf-482a-a5bd-345675adfad2" (UID: "2a9f4697-fabf-482a-a5bd-345675adfad2"). InnerVolumeSpecName "kube-api-access-5pbgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:16:11.464460 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.464439 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg"] Jul 08 09:16:11.472956 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:16:11.472936 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4759b3d_1c7d_4788_88d1_c8c4c7fb1385.slice/crio-a1623ee0592580ba59f0b9b49e08cbaebeecd0ec48cb46fcb7b51cdf576c63b7 WatchSource:0}: Error finding container a1623ee0592580ba59f0b9b49e08cbaebeecd0ec48cb46fcb7b51cdf576c63b7: Status 404 returned error can't find the container with id a1623ee0592580ba59f0b9b49e08cbaebeecd0ec48cb46fcb7b51cdf576c63b7 Jul 08 09:16:11.554917 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.554878 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gndlj\" (UniqueName: \"kubernetes.io/projected/e0d0c655-ce16-4b53-9e85-6a0341052c4f-kube-api-access-gndlj\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:16:11.555049 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.554923 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0d0c655-ce16-4b53-9e85-6a0341052c4f-extensions-socket-volume\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:16:11.555049 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.554937 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pbgh\" (UniqueName: \"kubernetes.io/projected/2a9f4697-fabf-482a-a5bd-345675adfad2-kube-api-access-5pbgh\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:16:11.795722 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.795627 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" event={"ID":"bde984ea-c427-45ba-b5a7-8771de4ab293","Type":"ContainerStarted","Data":"11ea0e3922bd9c90765d3605c4ba31ac677fbacf3563993dc49beec2173e8592"} Jul 08 09:16:11.796891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.796866 2572 generic.go:356] "Generic (PLEG): container finished" podID="2a9f4697-fabf-482a-a5bd-345675adfad2" containerID="2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc" exitCode=0 Jul 08 09:16:11.797006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.796914 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:11.797006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.796956 2572 scope.go:117] "RemoveContainer" containerID="2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc" Jul 08 09:16:11.798145 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.798117 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:16:11.798704 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.798677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" event={"ID":"ec78e5c0-6923-4824-bfd1-60c1b2d854d5","Type":"ContainerStarted","Data":"a502579b6260dd735e1ba72f486d808ac7361a71a884a0309b623bb0afd220f8"} Jul 08 09:16:11.798825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.798724 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" event={"ID":"ec78e5c0-6923-4824-bfd1-60c1b2d854d5","Type":"ContainerStarted","Data":"8002246b3426d5f71fc78c102716316f2ca153fc31d9ae0176d811f715d5f26f"} Jul 08 09:16:11.798825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.798745 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:16:11.800237 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.800209 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-r47dx\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:11.800546 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.800526 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" event={"ID":"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385","Type":"ContainerStarted","Data":"43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627"} Jul 08 09:16:11.800642 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.800567 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" event={"ID":"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385","Type":"ContainerStarted","Data":"a1623ee0592580ba59f0b9b49e08cbaebeecd0ec48cb46fcb7b51cdf576c63b7"} Jul 08 09:16:11.800642 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.800596 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:16:11.801822 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.801800 2572 generic.go:356] "Generic (PLEG): container finished" podID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" containerID="77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd" exitCode=0 Jul 08 09:16:11.801932 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.801826 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:16:11.807369 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.807348 2572 scope.go:117] "RemoveContainer" containerID="2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc" Jul 08 09:16:11.807762 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.807732 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc\": container with ID starting with 2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc not found: ID does not exist" containerID="2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc" Jul 08 09:16:11.807850 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.807763 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc"} err="failed to get container status \"2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc\": rpc error: code = NotFound desc = could not find container \"2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc\": container with ID starting with 2231d4790bc1b211fd0d0f9731404c163470b35ed27f91c5cf8d37690b3ba5cc not found: ID does not exist" Jul 08 09:16:11.807850 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.807786 2572 scope.go:117] "RemoveContainer" containerID="77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd" Jul 08 09:16:11.815755 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.815738 2572 scope.go:117] "RemoveContainer" containerID="77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd" Jul 08 09:16:11.816019 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.816001 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd\": container with ID starting with 77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd not found: ID does not exist" containerID="77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd" Jul 08 09:16:11.816075 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.816025 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd"} err="failed to get container status \"77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd\": rpc error: code = NotFound desc = could not find container \"77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd\": container with ID starting with 77cc8d2678da5c43fe53069dc7b16f09e9893ce5be07ec613c62e3502fc58afd not found: ID does not exist" Jul 08 09:16:11.819612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.819571 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fwnpr" podStartSLOduration=1.687552138 podStartE2EDuration="24.819558735s" totalImagesPullingTime="23.132006597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:11.817814785 +0000 UTC m=+550.172826352" watchObservedRunningTime="2026-07-08 09:16:11.819558735 +0000 UTC m=+550.174570296" Jul 08 09:16:11.821057 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.821032 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-r47dx\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:11.844141 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.844075 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" podStartSLOduration=13.844053098 podStartE2EDuration="13.844053098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:11.842397818 +0000 UTC m=+550.197409414" watchObservedRunningTime="2026-07-08 09:16:11.844053098 +0000 UTC m=+550.199064666" Jul 08 09:16:11.862075 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:11.862027 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" podStartSLOduration=13.862009795 podStartE2EDuration="13.862009795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:11.860732956 +0000 UTC m=+550.215744519" watchObservedRunningTime="2026-07-08 09:16:11.862009795 +0000 UTC m=+550.217021357" Jul 08 09:16:11.863609 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:11.863580 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:16:12.174067 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:12.174035 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ppv8x\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ppv8x" Jul 08 09:16:12.175788 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:12.175763 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-r47dx\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-r47dx" Jul 08 09:16:12.175894 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:12.175829 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9f4697-fabf-482a-a5bd-345675adfad2" path="/var/lib/kubelet/pods/2a9f4697-fabf-482a-a5bd-345675adfad2/volumes" Jul 08 09:16:12.179180 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:12.176715 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d0c655-ce16-4b53-9e85-6a0341052c4f" path="/var/lib/kubelet/pods/e0d0c655-ce16-4b53-9e85-6a0341052c4f/volumes" Jul 08 09:16:22.812580 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:22.812548 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:16:22.813078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:22.812624 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-47q2w" Jul 08 09:16:23.124705 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.124677 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2"] Jul 08 09:16:23.128691 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.128674 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.157115 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.157082 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2"] Jul 08 09:16:23.266874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.266837 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d49720a1-4354-4edb-a014-a9f2ed28cb20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9pzx2\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.266874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.266897 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rssk\" (UniqueName: \"kubernetes.io/projected/d49720a1-4354-4edb-a014-a9f2ed28cb20-kube-api-access-4rssk\") pod \"kuadrant-operator-controller-manager-86878c769f-9pzx2\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.370148 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.370107 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d49720a1-4354-4edb-a014-a9f2ed28cb20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9pzx2\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.370148 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.370153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rssk\" (UniqueName: \"kubernetes.io/projected/d49720a1-4354-4edb-a014-a9f2ed28cb20-kube-api-access-4rssk\") pod \"kuadrant-operator-controller-manager-86878c769f-9pzx2\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.370536 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.370516 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d49720a1-4354-4edb-a014-a9f2ed28cb20-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9pzx2\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.381106 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.381037 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rssk\" (UniqueName: \"kubernetes.io/projected/d49720a1-4354-4edb-a014-a9f2ed28cb20-kube-api-access-4rssk\") pod \"kuadrant-operator-controller-manager-86878c769f-9pzx2\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.440059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.440025 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.584301 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.584269 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2"] Jul 08 09:16:23.587745 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:16:23.587720 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49720a1_4354_4edb_a014_a9f2ed28cb20.slice/crio-a432bc548c7e270134503776c0cb475db84ed016309c70e2cf3923eb35c222e8 WatchSource:0}: Error finding container a432bc548c7e270134503776c0cb475db84ed016309c70e2cf3923eb35c222e8: Status 404 returned error can't find the container with id a432bc548c7e270134503776c0cb475db84ed016309c70e2cf3923eb35c222e8 Jul 08 09:16:23.867431 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.867390 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" event={"ID":"d49720a1-4354-4edb-a014-a9f2ed28cb20","Type":"ContainerStarted","Data":"83c745d788432e803479c2740b475670be6b3d044aab6eb22444d7279a711c2c"} Jul 08 09:16:23.867823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.867445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" event={"ID":"d49720a1-4354-4edb-a014-a9f2ed28cb20","Type":"ContainerStarted","Data":"a432bc548c7e270134503776c0cb475db84ed016309c70e2cf3923eb35c222e8"} Jul 08 09:16:23.867823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.867526 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:23.889003 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:23.888957 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" podStartSLOduration=0.888941958 podStartE2EDuration="888.941958ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:23.888087441 +0000 UTC m=+562.243099031" watchObservedRunningTime="2026-07-08 09:16:23.888941958 +0000 UTC m=+562.243953521" Jul 08 09:16:26.468693 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:26.468657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:26.571827 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:26.571795 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:34.875856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:34.875824 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:16:34.929336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:34.929308 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg"] Jul 08 09:16:34.929531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:34.929506 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" podUID="f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" containerName="manager" containerID="cri-o://43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627" gracePeriod=10 Jul 08 09:16:35.702316 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.702292 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:16:35.785519 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.785490 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42jzw\" (UniqueName: \"kubernetes.io/projected/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-kube-api-access-42jzw\") pod \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " Jul 08 09:16:35.785692 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.785565 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-extensions-socket-volume\") pod \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\" (UID: \"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385\") " Jul 08 09:16:35.785977 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.785952 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" (UID: "f4759b3d-1c7d-4788-88d1-c8c4c7fb1385"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:16:35.787541 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.787515 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-kube-api-access-42jzw" (OuterVolumeSpecName: "kube-api-access-42jzw") pod "f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" (UID: "f4759b3d-1c7d-4788-88d1-c8c4c7fb1385"). InnerVolumeSpecName "kube-api-access-42jzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:16:35.887027 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.886992 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42jzw\" (UniqueName: \"kubernetes.io/projected/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-kube-api-access-42jzw\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:16:35.887027 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.887020 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385-extensions-socket-volume\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:16:35.931285 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.931249 2572 generic.go:356] "Generic (PLEG): container finished" podID="f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" containerID="43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627" exitCode=0 Jul 08 09:16:35.931464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.931299 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" Jul 08 09:16:35.931464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.931343 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" event={"ID":"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385","Type":"ContainerDied","Data":"43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627"} Jul 08 09:16:35.931464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.931408 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg" event={"ID":"f4759b3d-1c7d-4788-88d1-c8c4c7fb1385","Type":"ContainerDied","Data":"a1623ee0592580ba59f0b9b49e08cbaebeecd0ec48cb46fcb7b51cdf576c63b7"} Jul 08 09:16:35.931464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.931438 2572 scope.go:117] "RemoveContainer" containerID="43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627" Jul 08 09:16:35.943662 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.943633 2572 scope.go:117] "RemoveContainer" containerID="43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627" Jul 08 09:16:35.944014 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:16:35.943992 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627\": container with ID starting with 43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627 not found: ID does not exist" containerID="43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627" Jul 08 09:16:35.944074 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.944025 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627"} err="failed to get container status \"43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627\": rpc error: code = NotFound desc = could not find container \"43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627\": container with ID starting with 43e527e27cadb3941cc2295ef922991e17116c90573504e332d0ae7c6911c627 not found: ID does not exist" Jul 08 09:16:35.954537 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.954514 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg"] Jul 08 09:16:35.960789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:35.960768 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rjtrg"] Jul 08 09:16:36.175821 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:36.175751 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" path="/var/lib/kubelet/pods/f4759b3d-1c7d-4788-88d1-c8c4c7fb1385/volumes" Jul 08 09:16:37.154514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:37.154480 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:39.335855 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.335824 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x"] Jul 08 09:16:39.336320 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.336304 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" containerName="manager" Jul 08 09:16:39.336367 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.336322 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" containerName="manager" Jul 08 09:16:39.336402 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.336379 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4759b3d-1c7d-4788-88d1-c8c4c7fb1385" containerName="manager" Jul 08 09:16:39.366311 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.366266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:39.471440 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.471412 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x"] Jul 08 09:16:39.471614 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.471536 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.474325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.474305 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-brfts\"" Jul 08 09:16:39.495652 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.495599 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:39.519802 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519781 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.519918 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519821 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.519918 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519842 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.519918 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519878 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8e350c-c0f2-4339-84b0-81dcb316e033-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.520042 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519933 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.520042 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.520042 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.519990 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjvhl\" (UniqueName: \"kubernetes.io/projected/5b8e350c-c0f2-4339-84b0-81dcb316e033-kube-api-access-vjvhl\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.520175 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.520079 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.520228 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.520167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621365 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621303 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621516 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621399 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621516 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621425 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621516 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8e350c-c0f2-4339-84b0-81dcb316e033-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621516 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621478 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621746 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621518 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621746 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjvhl\" (UniqueName: \"kubernetes.io/projected/5b8e350c-c0f2-4339-84b0-81dcb316e033-kube-api-access-vjvhl\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621746 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621676 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621746 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621728 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.621946 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.621798 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.622280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.622244 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.622280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.622271 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.622463 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.622312 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.622463 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.622341 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b8e350c-c0f2-4339-84b0-81dcb316e033-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.623994 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.623971 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.624090 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.624075 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.629859 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.629837 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b8e350c-c0f2-4339-84b0-81dcb316e033-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.630203 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.630184 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjvhl\" (UniqueName: \"kubernetes.io/projected/5b8e350c-c0f2-4339-84b0-81dcb316e033-kube-api-access-vjvhl\") pod \"maas-default-gateway-openshift-default-56654dcf7-xdj4x\" (UID: \"5b8e350c-c0f2-4339-84b0-81dcb316e033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.782891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.782855 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:39.955620 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.955591 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x"] Jul 08 09:16:39.958073 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:16:39.958048 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8e350c_c0f2_4339_84b0_81dcb316e033.slice/crio-c2ab8fe5bcbfda5bcf41a6917233c1cde121f77a72d75997c653d1317cd897c3 WatchSource:0}: Error finding container c2ab8fe5bcbfda5bcf41a6917233c1cde121f77a72d75997c653d1317cd897c3: Status 404 returned error can't find the container with id c2ab8fe5bcbfda5bcf41a6917233c1cde121f77a72d75997c653d1317cd897c3 Jul 08 09:16:39.960200 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.960169 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:16:39.960281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.960236 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:16:39.960281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:39.960267 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:16:40.961486 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:40.961445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" event={"ID":"5b8e350c-c0f2-4339-84b0-81dcb316e033","Type":"ContainerStarted","Data":"88e7cd52ffce44c14799e2707f283f55b069b774152fb5e230e797fd65b07019"} Jul 08 09:16:40.961962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:40.961525 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" event={"ID":"5b8e350c-c0f2-4339-84b0-81dcb316e033","Type":"ContainerStarted","Data":"c2ab8fe5bcbfda5bcf41a6917233c1cde121f77a72d75997c653d1317cd897c3"} Jul 08 09:16:40.984888 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:40.984837 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" podStartSLOduration=1.984822023 podStartE2EDuration="1.984822023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:16:40.983093097 +0000 UTC m=+579.338104661" watchObservedRunningTime="2026-07-08 09:16:40.984822023 +0000 UTC m=+579.339833568" Jul 08 09:16:41.783993 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:41.783953 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:41.789138 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:41.789116 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:41.967572 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:41.967541 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:41.968624 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:41.968602 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:16:42.012094 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:42.012052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:42.130552 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:42.130513 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:42.231639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:42.231567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:42.850519 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:42.850483 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:16:42.922851 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:16:42.922818 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:03.669250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:03.669195 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:15.076273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:15.076239 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:15.194514 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:15.194445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:21.203261 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.203220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:21.211499 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.211466 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7745ff9c84-gd8b5"] Jul 08 09:17:21.215059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.215038 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.218206 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.218187 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 09:17:21.218206 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.218198 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-phsjl\"" Jul 08 09:17:21.218441 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.218427 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 09:17:21.235869 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.235845 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7745ff9c84-gd8b5"] Jul 08 09:17:21.307671 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.307609 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:21.319757 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.319734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.319879 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.319850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wx2n\" (UniqueName: \"kubernetes.io/projected/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-kube-api-access-9wx2n\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.390489 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.390455 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:21.394886 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.394865 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-jdmtd"] Jul 08 09:17:21.397545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.397527 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:21.411463 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.411440 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-jdmtd"] Jul 08 09:17:21.420855 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.420833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nftcz\" (UniqueName: \"kubernetes.io/projected/27d64409-da14-4b56-aa7e-4318ef37fe56-kube-api-access-nftcz\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:21.420983 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.420889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wx2n\" (UniqueName: \"kubernetes.io/projected/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-kube-api-access-9wx2n\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.420983 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.420958 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:21.421068 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.421060 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.421174 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:17:21.421160 2572 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 08 09:17:21.421237 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:17:21.421226 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert podName:b3930ad8-4a4b-4619-9a36-8ef06a5adc93 nodeName:}" failed. No retries permitted until 2026-07-08 09:17:21.921205692 +0000 UTC m=+620.276217249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert") pod "maas-controller-7745ff9c84-gd8b5" (UID: "b3930ad8-4a4b-4619-9a36-8ef06a5adc93") : secret "maas-controller-webhook-cert" not found Jul 08 09:17:21.432436 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.432411 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wx2n\" (UniqueName: \"kubernetes.io/projected/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-kube-api-access-9wx2n\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.521925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.521784 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nftcz\" (UniqueName: \"kubernetes.io/projected/27d64409-da14-4b56-aa7e-4318ef37fe56-kube-api-access-nftcz\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:21.521925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.521845 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:21.522182 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:17:21.521955 2572 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 08 09:17:21.522182 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:17:21.522016 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert podName:27d64409-da14-4b56-aa7e-4318ef37fe56 nodeName:}" failed. No retries permitted until 2026-07-08 09:17:22.022002664 +0000 UTC m=+620.377014204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert") pod "maas-controller-d6cbf4fcb-jdmtd" (UID: "27d64409-da14-4b56-aa7e-4318ef37fe56") : secret "maas-controller-webhook-cert" not found Jul 08 09:17:21.530496 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.530473 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nftcz\" (UniqueName: \"kubernetes.io/projected/27d64409-da14-4b56-aa7e-4318ef37fe56-kube-api-access-nftcz\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:21.661716 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.661681 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:21.926087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.926053 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:21.928427 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:21.928398 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert\") pod \"maas-controller-7745ff9c84-gd8b5\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:22.027459 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:22.027427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:22.029662 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:22.029645 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert\") pod \"maas-controller-d6cbf4fcb-jdmtd\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:22.125790 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:22.125753 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:22.263565 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:22.263536 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7745ff9c84-gd8b5"] Jul 08 09:17:22.265781 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:17:22.265756 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3930ad8_4a4b_4619_9a36_8ef06a5adc93.slice/crio-c36421844eb17440222c6cf883870234d8ba7349a52dc3924bb3d2aedec12129 WatchSource:0}: Error finding container c36421844eb17440222c6cf883870234d8ba7349a52dc3924bb3d2aedec12129: Status 404 returned error can't find the container with id c36421844eb17440222c6cf883870234d8ba7349a52dc3924bb3d2aedec12129 Jul 08 09:17:22.308061 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:22.308039 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:22.446038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:22.446008 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-jdmtd"] Jul 08 09:17:22.447691 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:17:22.447664 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d64409_da14_4b56_aa7e_4318ef37fe56.slice/crio-f45cf3fa65c844e67bd9c091dd1912909d7e8826ca355b68c03dc1571400defd WatchSource:0}: Error finding container f45cf3fa65c844e67bd9c091dd1912909d7e8826ca355b68c03dc1571400defd: Status 404 returned error can't find the container with id f45cf3fa65c844e67bd9c091dd1912909d7e8826ca355b68c03dc1571400defd Jul 08 09:17:23.190770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:23.190680 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" event={"ID":"27d64409-da14-4b56-aa7e-4318ef37fe56","Type":"ContainerStarted","Data":"f45cf3fa65c844e67bd9c091dd1912909d7e8826ca355b68c03dc1571400defd"} Jul 08 09:17:23.195140 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:23.195102 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" event={"ID":"b3930ad8-4a4b-4619-9a36-8ef06a5adc93","Type":"ContainerStarted","Data":"c36421844eb17440222c6cf883870234d8ba7349a52dc3924bb3d2aedec12129"} Jul 08 09:17:25.207360 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.207323 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" event={"ID":"27d64409-da14-4b56-aa7e-4318ef37fe56","Type":"ContainerStarted","Data":"1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15"} Jul 08 09:17:25.207799 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.207511 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:25.208869 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.208843 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" event={"ID":"b3930ad8-4a4b-4619-9a36-8ef06a5adc93","Type":"ContainerStarted","Data":"52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b"} Jul 08 09:17:25.209014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.208951 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:25.249811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.249759 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" podStartSLOduration=1.608092962 podStartE2EDuration="4.24974228s" totalImagesPullingTime="2.641649318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:17:25.249252121 +0000 UTC m=+623.604263684" watchObservedRunningTime="2026-07-08 09:17:25.24974228 +0000 UTC m=+623.604753843" Jul 08 09:17:25.281334 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.281281 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" podStartSLOduration=1.4475667190000001 podStartE2EDuration="4.281266147s" totalImagesPullingTime="2.833699428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:17:25.277336409 +0000 UTC m=+623.632347973" watchObservedRunningTime="2026-07-08 09:17:25.281266147 +0000 UTC m=+623.636277709" Jul 08 09:17:25.404329 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.404292 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:25.545273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:25.545194 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:36.219757 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.219727 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:36.220170 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.219884 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:17:36.284942 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.284893 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7745ff9c84-gd8b5"] Jul 08 09:17:36.285167 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.285139 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" podUID="b3930ad8-4a4b-4619-9a36-8ef06a5adc93" containerName="manager" containerID="cri-o://52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b" gracePeriod=30 Jul 08 09:17:36.531039 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.531014 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:36.680245 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.680215 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wx2n\" (UniqueName: \"kubernetes.io/projected/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-kube-api-access-9wx2n\") pod \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " Jul 08 09:17:36.680446 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.680289 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert\") pod \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\" (UID: \"b3930ad8-4a4b-4619-9a36-8ef06a5adc93\") " Jul 08 09:17:36.682534 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.682506 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-kube-api-access-9wx2n" (OuterVolumeSpecName: "kube-api-access-9wx2n") pod "b3930ad8-4a4b-4619-9a36-8ef06a5adc93" (UID: "b3930ad8-4a4b-4619-9a36-8ef06a5adc93"). InnerVolumeSpecName "kube-api-access-9wx2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:17:36.682639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.682569 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b3930ad8-4a4b-4619-9a36-8ef06a5adc93" (UID: "b3930ad8-4a4b-4619-9a36-8ef06a5adc93"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:17:36.781161 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.781073 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-webhook-cert\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:17:36.781161 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:36.781102 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wx2n\" (UniqueName: \"kubernetes.io/projected/b3930ad8-4a4b-4619-9a36-8ef06a5adc93-kube-api-access-9wx2n\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:17:37.271215 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.271180 2572 generic.go:356] "Generic (PLEG): container finished" podID="b3930ad8-4a4b-4619-9a36-8ef06a5adc93" containerID="52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b" exitCode=0 Jul 08 09:17:37.271602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.271222 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" Jul 08 09:17:37.271602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.271264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" event={"ID":"b3930ad8-4a4b-4619-9a36-8ef06a5adc93","Type":"ContainerDied","Data":"52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b"} Jul 08 09:17:37.271602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.271318 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7745ff9c84-gd8b5" event={"ID":"b3930ad8-4a4b-4619-9a36-8ef06a5adc93","Type":"ContainerDied","Data":"c36421844eb17440222c6cf883870234d8ba7349a52dc3924bb3d2aedec12129"} Jul 08 09:17:37.271602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.271344 2572 scope.go:117] "RemoveContainer" containerID="52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b" Jul 08 09:17:37.280744 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.280727 2572 scope.go:117] "RemoveContainer" containerID="52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b" Jul 08 09:17:37.281076 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:17:37.281058 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b\": container with ID starting with 52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b not found: ID does not exist" containerID="52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b" Jul 08 09:17:37.281154 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.281083 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b"} err="failed to get container status \"52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b\": rpc error: code = NotFound desc = could not find container \"52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b\": container with ID starting with 52e877cff6f97798ecb8b9f8ca3f0400d56056357d13822689ea69000cdd5e2b not found: ID does not exist" Jul 08 09:17:37.293908 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.293884 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7745ff9c84-gd8b5"] Jul 08 09:17:37.299783 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:37.299764 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7745ff9c84-gd8b5"] Jul 08 09:17:38.175659 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:38.175624 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3930ad8-4a4b-4619-9a36-8ef06a5adc93" path="/var/lib/kubelet/pods/b3930ad8-4a4b-4619-9a36-8ef06a5adc93/volumes" Jul 08 09:17:56.024570 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:56.024490 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:56.511566 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:56.511519 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:57.001368 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.001323 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:57.147038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.146995 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:17:57.261202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.261125 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6hj89"] Jul 08 09:17:57.261570 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.261554 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3930ad8-4a4b-4619-9a36-8ef06a5adc93" containerName="manager" Jul 08 09:17:57.261615 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.261571 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3930ad8-4a4b-4619-9a36-8ef06a5adc93" containerName="manager" Jul 08 09:17:57.261658 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.261645 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3930ad8-4a4b-4619-9a36-8ef06a5adc93" containerName="manager" Jul 08 09:17:57.264664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.264644 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.267189 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.267168 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 09:17:57.267189 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.267190 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7985j\"" Jul 08 09:17:57.277010 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.276990 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6hj89"] Jul 08 09:17:57.373834 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.373801 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce96574-14b8-488d-86e2-a1ed910f1a31-plugins-config-volume\") pod \"payload-processing-58f86c7798-6hj89\" (UID: \"7ce96574-14b8-488d-86e2-a1ed910f1a31\") " pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.374022 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.373969 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh26k\" (UniqueName: \"kubernetes.io/projected/7ce96574-14b8-488d-86e2-a1ed910f1a31-kube-api-access-gh26k\") pod \"payload-processing-58f86c7798-6hj89\" (UID: \"7ce96574-14b8-488d-86e2-a1ed910f1a31\") " pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.474843 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.474806 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh26k\" (UniqueName: \"kubernetes.io/projected/7ce96574-14b8-488d-86e2-a1ed910f1a31-kube-api-access-gh26k\") pod \"payload-processing-58f86c7798-6hj89\" (UID: \"7ce96574-14b8-488d-86e2-a1ed910f1a31\") " pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.475060 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.474886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce96574-14b8-488d-86e2-a1ed910f1a31-plugins-config-volume\") pod \"payload-processing-58f86c7798-6hj89\" (UID: \"7ce96574-14b8-488d-86e2-a1ed910f1a31\") " pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.475553 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.475534 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce96574-14b8-488d-86e2-a1ed910f1a31-plugins-config-volume\") pod \"payload-processing-58f86c7798-6hj89\" (UID: \"7ce96574-14b8-488d-86e2-a1ed910f1a31\") " pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.482620 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.482594 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh26k\" (UniqueName: \"kubernetes.io/projected/7ce96574-14b8-488d-86e2-a1ed910f1a31-kube-api-access-gh26k\") pod \"payload-processing-58f86c7798-6hj89\" (UID: \"7ce96574-14b8-488d-86e2-a1ed910f1a31\") " pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.576289 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.576205 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:17:57.736491 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:57.736466 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6hj89"] Jul 08 09:17:57.739139 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:17:57.739109 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce96574_14b8_488d_86e2_a1ed910f1a31.slice/crio-819aa28bbf084188aa09a3818e14e07f5284b4a3d607b2a09eb1bc9b7ba0d677 WatchSource:0}: Error finding container 819aa28bbf084188aa09a3818e14e07f5284b4a3d607b2a09eb1bc9b7ba0d677: Status 404 returned error can't find the container with id 819aa28bbf084188aa09a3818e14e07f5284b4a3d607b2a09eb1bc9b7ba0d677 Jul 08 09:17:58.382500 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:17:58.382450 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" event={"ID":"7ce96574-14b8-488d-86e2-a1ed910f1a31","Type":"ContainerStarted","Data":"819aa28bbf084188aa09a3818e14e07f5284b4a3d607b2a09eb1bc9b7ba0d677"} Jul 08 09:18:00.395365 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:00.395325 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" event={"ID":"7ce96574-14b8-488d-86e2-a1ed910f1a31","Type":"ContainerStarted","Data":"30cf4d84c17f6b113172a636f367da05f9e816b4d3d0c52aa9933e67886a9dbe"} Jul 08 09:18:00.395751 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:00.395402 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:18:00.412989 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:00.412936 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" podStartSLOduration=1.800377262 podStartE2EDuration="3.412917047s" totalImagesPullingTime="1.612539785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:18:00.412234648 +0000 UTC m=+658.767246210" watchObservedRunningTime="2026-07-08 09:18:00.412917047 +0000 UTC m=+658.767928606" Jul 08 09:18:05.422875 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:05.422843 2572 generic.go:356] "Generic (PLEG): container finished" podID="5b8e350c-c0f2-4339-84b0-81dcb316e033" containerID="88e7cd52ffce44c14799e2707f283f55b069b774152fb5e230e797fd65b07019" exitCode=0 Jul 08 09:18:05.423313 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:05.422938 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" event={"ID":"5b8e350c-c0f2-4339-84b0-81dcb316e033","Type":"ContainerDied","Data":"88e7cd52ffce44c14799e2707f283f55b069b774152fb5e230e797fd65b07019"} Jul 08 09:18:05.423358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:05.423347 2572 scope.go:117] "RemoveContainer" containerID="88e7cd52ffce44c14799e2707f283f55b069b774152fb5e230e797fd65b07019" Jul 08 09:18:05.424428 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:05.424393 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:18:05.424499 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:05.424483 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:18:05.424558 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:05.424529 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:18:06.454391 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:06.454354 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" event={"ID":"5b8e350c-c0f2-4339-84b0-81dcb316e033","Type":"ContainerStarted","Data":"54c085bde691c9b432f6c3cde9d7116f8b4af480117a08eccd0ccc4dd4c8df1f"} Jul 08 09:18:06.783269 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:06.783168 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:18:06.783423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:06.783347 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xdj4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 09:18:06.783423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:06.783395 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" podUID="5b8e350c-c0f2-4339-84b0-81dcb316e033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 09:18:07.006960 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:07.006925 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:07.783599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:07.783562 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xdj4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 09:18:07.784059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:07.783616 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" podUID="5b8e350c-c0f2-4339-84b0-81dcb316e033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 09:18:08.492368 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:08.492326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:08.783404 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:08.783318 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xdj4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 09:18:08.783404 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:08.783394 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" podUID="5b8e350c-c0f2-4339-84b0-81dcb316e033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 09:18:09.783396 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:09.783361 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xdj4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 08 09:18:09.783790 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:09.783419 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" podUID="5b8e350c-c0f2-4339-84b0-81dcb316e033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 08 09:18:10.788038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:10.788010 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:18:10.788438 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:10.788284 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:18:10.788922 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:10.788889 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xdj4x" Jul 08 09:18:11.402998 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:11.402970 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-6hj89" Jul 08 09:18:17.122330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.122299 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc"] Jul 08 09:18:17.133284 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.133255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.136856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.136821 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 09:18:17.136997 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.136856 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w2rf8\"" Jul 08 09:18:17.136997 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.136880 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 09:18:17.136997 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.136821 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 09:18:17.146544 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.142220 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc"] Jul 08 09:18:17.168642 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.168606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:17.267930 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.267872 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.268128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.267971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.268128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.268004 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.268128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.268036 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.268301 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.268132 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.268301 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.268266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfjk5\" (UniqueName: \"kubernetes.io/projected/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-kube-api-access-sfjk5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.369700 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.369659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.369891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.369741 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfjk5\" (UniqueName: \"kubernetes.io/projected/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-kube-api-access-sfjk5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.369891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.369794 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.369891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.369835 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.369891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.369862 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.369891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.369892 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.370199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.370148 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.370298 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.370202 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.370358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.370322 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.372259 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.372233 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.372408 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.372337 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.379255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.379230 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfjk5\" (UniqueName: \"kubernetes.io/projected/f3251e3d-e7be-4126-adca-d1c0ca0d0cab-kube-api-access-sfjk5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc\" (UID: \"f3251e3d-e7be-4126-adca-d1c0ca0d0cab\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.448491 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.448454 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:17.597624 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:17.597601 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc"] Jul 08 09:18:17.600247 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:18:17.600214 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3251e3d_e7be_4126_adca_d1c0ca0d0cab.slice/crio-c6ee29fc598981496f7f844a1fc5ac14efb4da55540d2dce3bfdd99534495bb5 WatchSource:0}: Error finding container c6ee29fc598981496f7f844a1fc5ac14efb4da55540d2dce3bfdd99534495bb5: Status 404 returned error can't find the container with id c6ee29fc598981496f7f844a1fc5ac14efb4da55540d2dce3bfdd99534495bb5 Jul 08 09:18:18.526616 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:18.526582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" event={"ID":"f3251e3d-e7be-4126-adca-d1c0ca0d0cab","Type":"ContainerStarted","Data":"c6ee29fc598981496f7f844a1fc5ac14efb4da55540d2dce3bfdd99534495bb5"} Jul 08 09:18:19.534851 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:19.534811 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" event={"ID":"f3251e3d-e7be-4126-adca-d1c0ca0d0cab","Type":"ContainerStarted","Data":"c30fc3532983b45b6a92e55e6ba76f36d870e6d682556230bc5a910a78790d13"} Jul 08 09:18:19.554012 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:19.553963 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" podStartSLOduration=1.343313065 podStartE2EDuration="2.553948411s" totalImagesPullingTime="1.210635346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:18:19.551168898 +0000 UTC m=+677.906180464" watchObservedRunningTime="2026-07-08 09:18:19.553948411 +0000 UTC m=+677.908959972" Jul 08 09:18:20.312315 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:20.312254 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:24.180776 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.180739 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g"] Jul 08 09:18:24.183628 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.183607 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.186189 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.186169 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 09:18:24.195167 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.195141 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g"] Jul 08 09:18:24.219420 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.219390 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:24.348757 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.348723 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.348968 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.348785 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.348968 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.348873 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tq9m\" (UniqueName: \"kubernetes.io/projected/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-kube-api-access-5tq9m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.348968 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.348948 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.349133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.349001 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.349133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.349040 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.450613 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.450523 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tq9m\" (UniqueName: \"kubernetes.io/projected/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-kube-api-access-5tq9m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.450613 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.450580 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.450836 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.450614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.450836 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.450637 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.450836 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.450735 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.450836 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.450797 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.451088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.451026 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.451088 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.451062 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.451199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.451185 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.453010 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.452988 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.453262 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.453243 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.460461 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.460436 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tq9m\" (UniqueName: \"kubernetes.io/projected/794a4e41-fbe5-4857-8f6c-aa48e4d64f66-kube-api-access-5tq9m\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2tm9g\" (UID: \"794a4e41-fbe5-4857-8f6c-aa48e4d64f66\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.494435 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.494409 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:24.636250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.636227 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g"] Jul 08 09:18:24.638994 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:18:24.638962 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794a4e41_fbe5_4857_8f6c_aa48e4d64f66.slice/crio-d7a070f510c3fa984cf1ca2321beea7bfc6885b098804a42e1ade138225f53d6 WatchSource:0}: Error finding container d7a070f510c3fa984cf1ca2321beea7bfc6885b098804a42e1ade138225f53d6: Status 404 returned error can't find the container with id d7a070f510c3fa984cf1ca2321beea7bfc6885b098804a42e1ade138225f53d6 Jul 08 09:18:24.640633 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:24.640611 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:18:25.571430 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:25.571391 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" event={"ID":"794a4e41-fbe5-4857-8f6c-aa48e4d64f66","Type":"ContainerStarted","Data":"fabbdb5f525614f2ace5a588828efc3d55c89a6d4b25c7f4b7a748f5916a0aac"} Jul 08 09:18:25.571891 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:25.571446 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" event={"ID":"794a4e41-fbe5-4857-8f6c-aa48e4d64f66","Type":"ContainerStarted","Data":"d7a070f510c3fa984cf1ca2321beea7bfc6885b098804a42e1ade138225f53d6"} Jul 08 09:18:25.589932 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:25.589743 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" podStartSLOduration=1.339185697 podStartE2EDuration="1.589724728s" totalImagesPullingTime="250.539031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:18:25.589365302 +0000 UTC m=+683.944376866" watchObservedRunningTime="2026-07-08 09:18:25.589724728 +0000 UTC m=+683.944736287" Jul 08 09:18:27.449527 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:27.449484 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:27.449957 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:27.449562 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:27.462433 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:27.462405 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:27.592623 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:27.592583 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc" Jul 08 09:18:34.494630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:34.494588 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:34.495040 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:34.494665 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:34.506943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:34.506921 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:34.633394 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:34.633368 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2tm9g" Jul 08 09:18:42.373383 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:42.373344 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:43.373340 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:43.373297 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:43.471004 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:43.470969 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:52.117409 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:52.117374 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:18:57.585243 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:18:57.585161 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:03.859641 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:03.859588 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:44.070142 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:44.070105 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:44.175304 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:44.175268 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:44.747222 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:44.747190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:44.891098 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:44.891059 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:45.924683 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:45.924614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:19:46.058361 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:19:46.058327 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:20:17.754603 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:17.754570 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:20:17.855185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:17.855141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:20:56.805761 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:56.805703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:20:56.911108 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:56.911076 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-jdmtd"] Jul 08 09:20:56.911329 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:56.911304 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" podUID="27d64409-da14-4b56-aa7e-4318ef37fe56" containerName="manager" containerID="cri-o://1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15" gracePeriod=30 Jul 08 09:20:57.162017 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.161990 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:20:57.170511 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.170489 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert\") pod \"27d64409-da14-4b56-aa7e-4318ef37fe56\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " Jul 08 09:20:57.170662 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.170630 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nftcz\" (UniqueName: \"kubernetes.io/projected/27d64409-da14-4b56-aa7e-4318ef37fe56-kube-api-access-nftcz\") pod \"27d64409-da14-4b56-aa7e-4318ef37fe56\" (UID: \"27d64409-da14-4b56-aa7e-4318ef37fe56\") " Jul 08 09:20:57.172453 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.172431 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "27d64409-da14-4b56-aa7e-4318ef37fe56" (UID: "27d64409-da14-4b56-aa7e-4318ef37fe56"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:20:57.172567 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.172545 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d64409-da14-4b56-aa7e-4318ef37fe56-kube-api-access-nftcz" (OuterVolumeSpecName: "kube-api-access-nftcz") pod "27d64409-da14-4b56-aa7e-4318ef37fe56" (UID: "27d64409-da14-4b56-aa7e-4318ef37fe56"). InnerVolumeSpecName "kube-api-access-nftcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:20:57.272089 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.272061 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nftcz\" (UniqueName: \"kubernetes.io/projected/27d64409-da14-4b56-aa7e-4318ef37fe56-kube-api-access-nftcz\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:20:57.272089 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.272090 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27d64409-da14-4b56-aa7e-4318ef37fe56-webhook-cert\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:20:57.413864 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.413831 2572 generic.go:356] "Generic (PLEG): container finished" podID="27d64409-da14-4b56-aa7e-4318ef37fe56" containerID="1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15" exitCode=0 Jul 08 09:20:57.414061 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.413879 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" Jul 08 09:20:57.414061 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.413930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" event={"ID":"27d64409-da14-4b56-aa7e-4318ef37fe56","Type":"ContainerDied","Data":"1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15"} Jul 08 09:20:57.414061 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.414013 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6cbf4fcb-jdmtd" event={"ID":"27d64409-da14-4b56-aa7e-4318ef37fe56","Type":"ContainerDied","Data":"f45cf3fa65c844e67bd9c091dd1912909d7e8826ca355b68c03dc1571400defd"} Jul 08 09:20:57.414061 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.414048 2572 scope.go:117] "RemoveContainer" containerID="1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15" Jul 08 09:20:57.423665 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.423647 2572 scope.go:117] "RemoveContainer" containerID="1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15" Jul 08 09:20:57.424033 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:20:57.424010 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15\": container with ID starting with 1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15 not found: ID does not exist" containerID="1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15" Jul 08 09:20:57.424102 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.424047 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15"} err="failed to get container status \"1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15\": rpc error: code = NotFound desc = could not find container \"1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15\": container with ID starting with 1f55fad58da3fc5db6219d6103fb59078735f028470f54ada7ed09c5a500ca15 not found: ID does not exist" Jul 08 09:20:57.440602 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.440577 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-jdmtd"] Jul 08 09:20:57.448339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:57.448320 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-jdmtd"] Jul 08 09:20:58.143032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.142973 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:20:58.176678 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.176648 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d64409-da14-4b56-aa7e-4318ef37fe56" path="/var/lib/kubelet/pods/27d64409-da14-4b56-aa7e-4318ef37fe56/volumes" Jul 08 09:20:58.257229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.257196 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-gmp4d"] Jul 08 09:20:58.257647 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.257635 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d64409-da14-4b56-aa7e-4318ef37fe56" containerName="manager" Jul 08 09:20:58.257695 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.257650 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d64409-da14-4b56-aa7e-4318ef37fe56" containerName="manager" Jul 08 09:20:58.257742 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.257711 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d64409-da14-4b56-aa7e-4318ef37fe56" containerName="manager" Jul 08 09:20:58.260873 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.260857 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.263640 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.263622 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 09:20:58.264619 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.264605 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-phsjl\"" Jul 08 09:20:58.264698 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.264619 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 09:20:58.280328 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.280306 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63-webhook-cert\") pod \"maas-controller-d6cbf4fcb-gmp4d\" (UID: \"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63\") " pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.280428 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.280356 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z9mr\" (UniqueName: \"kubernetes.io/projected/48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63-kube-api-access-8z9mr\") pod \"maas-controller-d6cbf4fcb-gmp4d\" (UID: \"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63\") " pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.283710 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.283689 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-gmp4d"] Jul 08 09:20:58.380980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.380952 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63-webhook-cert\") pod \"maas-controller-d6cbf4fcb-gmp4d\" (UID: \"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63\") " pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.381153 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.381006 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z9mr\" (UniqueName: \"kubernetes.io/projected/48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63-kube-api-access-8z9mr\") pod \"maas-controller-d6cbf4fcb-gmp4d\" (UID: \"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63\") " pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.383474 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.383441 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63-webhook-cert\") pod \"maas-controller-d6cbf4fcb-gmp4d\" (UID: \"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63\") " pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.389709 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.389686 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z9mr\" (UniqueName: \"kubernetes.io/projected/48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63-kube-api-access-8z9mr\") pod \"maas-controller-d6cbf4fcb-gmp4d\" (UID: \"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63\") " pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.571846 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.571750 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:58.719963 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:20:58.719930 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d8f8e9_dc1e_4d4f_ac9e_2907ff04cf63.slice/crio-a5cadcd0b25c397025118a06352ad9203ebc2280b0e58e84258f6f775d4edd96 WatchSource:0}: Error finding container a5cadcd0b25c397025118a06352ad9203ebc2280b0e58e84258f6f775d4edd96: Status 404 returned error can't find the container with id a5cadcd0b25c397025118a06352ad9203ebc2280b0e58e84258f6f775d4edd96 Jul 08 09:20:58.721035 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:58.721011 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6cbf4fcb-gmp4d"] Jul 08 09:20:59.427954 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:59.427852 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" event={"ID":"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63","Type":"ContainerStarted","Data":"adcdbee2937eef0fe1dea2b6b8e5a7203ad981e70c3b9069cdbe45fa16cc2124"} Jul 08 09:20:59.427954 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:59.427936 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" event={"ID":"48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63","Type":"ContainerStarted","Data":"a5cadcd0b25c397025118a06352ad9203ebc2280b0e58e84258f6f775d4edd96"} Jul 08 09:20:59.428348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:59.427967 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:20:59.451198 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:20:59.451150 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" podStartSLOduration=1.033159932 podStartE2EDuration="1.451133773s" totalImagesPullingTime="417.973841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:20:59.44963326 +0000 UTC m=+837.804644860" watchObservedRunningTime="2026-07-08 09:20:59.451133773 +0000 UTC m=+837.806145334" Jul 08 09:21:10.441693 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:10.441665 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6cbf4fcb-gmp4d" Jul 08 09:21:41.264128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:41.264081 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:41.391727 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:41.391684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:41.533190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:41.533096 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:44.470236 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:44.470201 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:44.570848 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:44.570815 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:47.556403 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:47.556362 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:47.661498 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:47.661455 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:47.929486 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:47.929447 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:48.028361 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:48.028326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:53.615273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:53.615186 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:21:53.711281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:21:53.711250 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:23:29.885018 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:23:29.884924 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:25:03.444921 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:03.444859 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:25:03.501156 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:03.501123 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:25:12.044635 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:12.044602 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:25:12.143334 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:12.143295 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:25:40.048624 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:40.048584 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:25:48.636272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:48.636240 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2"] Jul 08 09:25:48.636775 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:48.636443 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" podUID="d49720a1-4354-4edb-a014-a9f2ed28cb20" containerName="manager" containerID="cri-o://83c745d788432e803479c2740b475670be6b3d044aab6eb22444d7279a711c2c" gracePeriod=10 Jul 08 09:25:50.997934 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:50.997885 2572 generic.go:356] "Generic (PLEG): container finished" podID="d49720a1-4354-4edb-a014-a9f2ed28cb20" containerID="83c745d788432e803479c2740b475670be6b3d044aab6eb22444d7279a711c2c" exitCode=0 Jul 08 09:25:50.998221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:50.997935 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" event={"ID":"d49720a1-4354-4edb-a014-a9f2ed28cb20","Type":"ContainerDied","Data":"83c745d788432e803479c2740b475670be6b3d044aab6eb22444d7279a711c2c"} Jul 08 09:25:50.998221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:50.997989 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" event={"ID":"d49720a1-4354-4edb-a014-a9f2ed28cb20","Type":"ContainerDied","Data":"a432bc548c7e270134503776c0cb475db84ed016309c70e2cf3923eb35c222e8"} Jul 08 09:25:50.998221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:50.998006 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a432bc548c7e270134503776c0cb475db84ed016309c70e2cf3923eb35c222e8" Jul 08 09:25:50.999120 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:50.999106 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:25:51.073842 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:51.073810 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d49720a1-4354-4edb-a014-a9f2ed28cb20-extensions-socket-volume\") pod \"d49720a1-4354-4edb-a014-a9f2ed28cb20\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " Jul 08 09:25:51.074010 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:51.073957 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rssk\" (UniqueName: \"kubernetes.io/projected/d49720a1-4354-4edb-a014-a9f2ed28cb20-kube-api-access-4rssk\") pod \"d49720a1-4354-4edb-a014-a9f2ed28cb20\" (UID: \"d49720a1-4354-4edb-a014-a9f2ed28cb20\") " Jul 08 09:25:51.074199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:51.074178 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49720a1-4354-4edb-a014-a9f2ed28cb20-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d49720a1-4354-4edb-a014-a9f2ed28cb20" (UID: "d49720a1-4354-4edb-a014-a9f2ed28cb20"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:25:51.075856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:51.075833 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49720a1-4354-4edb-a014-a9f2ed28cb20-kube-api-access-4rssk" (OuterVolumeSpecName: "kube-api-access-4rssk") pod "d49720a1-4354-4edb-a014-a9f2ed28cb20" (UID: "d49720a1-4354-4edb-a014-a9f2ed28cb20"). InnerVolumeSpecName "kube-api-access-4rssk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:25:51.175025 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:51.174995 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rssk\" (UniqueName: \"kubernetes.io/projected/d49720a1-4354-4edb-a014-a9f2ed28cb20-kube-api-access-4rssk\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:25:51.175025 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:51.175022 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d49720a1-4354-4edb-a014-a9f2ed28cb20-extensions-socket-volume\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:25:52.003811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:52.003779 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2" Jul 08 09:25:52.027323 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:52.027295 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2"] Jul 08 09:25:52.033277 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:52.033253 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9pzx2"] Jul 08 09:25:52.176169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:25:52.176134 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49720a1-4354-4edb-a014-a9f2ed28cb20" path="/var/lib/kubelet/pods/d49720a1-4354-4edb-a014-a9f2ed28cb20/volumes" Jul 08 09:26:11.324833 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:11.324800 2572 scope.go:117] "RemoveContainer" containerID="83c745d788432e803479c2740b475670be6b3d044aab6eb22444d7279a711c2c" Jul 08 09:26:49.829579 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.829496 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv"] Jul 08 09:26:49.830054 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.829978 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49720a1-4354-4edb-a014-a9f2ed28cb20" containerName="manager" Jul 08 09:26:49.830054 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.829990 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49720a1-4354-4edb-a014-a9f2ed28cb20" containerName="manager" Jul 08 09:26:49.830131 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.830073 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49720a1-4354-4edb-a014-a9f2ed28cb20" containerName="manager" Jul 08 09:26:49.833473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.833455 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:49.836187 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.836168 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2tbss\"" Jul 08 09:26:49.848623 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.848596 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv"] Jul 08 09:26:49.944390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.944357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkrlp\" (UniqueName: \"kubernetes.io/projected/13e07f80-e221-4b96-a6ff-03d15f3700d2-kube-api-access-zkrlp\") pod \"kuadrant-operator-controller-manager-86878c769f-rrtqv\" (UID: \"13e07f80-e221-4b96-a6ff-03d15f3700d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:49.944579 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:49.944425 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13e07f80-e221-4b96-a6ff-03d15f3700d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrtqv\" (UID: \"13e07f80-e221-4b96-a6ff-03d15f3700d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:50.045039 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.045003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13e07f80-e221-4b96-a6ff-03d15f3700d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrtqv\" (UID: \"13e07f80-e221-4b96-a6ff-03d15f3700d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:50.045218 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.045110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkrlp\" (UniqueName: \"kubernetes.io/projected/13e07f80-e221-4b96-a6ff-03d15f3700d2-kube-api-access-zkrlp\") pod \"kuadrant-operator-controller-manager-86878c769f-rrtqv\" (UID: \"13e07f80-e221-4b96-a6ff-03d15f3700d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:50.045380 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.045359 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13e07f80-e221-4b96-a6ff-03d15f3700d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rrtqv\" (UID: \"13e07f80-e221-4b96-a6ff-03d15f3700d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:50.058042 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.058016 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkrlp\" (UniqueName: \"kubernetes.io/projected/13e07f80-e221-4b96-a6ff-03d15f3700d2-kube-api-access-zkrlp\") pod \"kuadrant-operator-controller-manager-86878c769f-rrtqv\" (UID: \"13e07f80-e221-4b96-a6ff-03d15f3700d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:50.145537 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.145511 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:50.298235 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:26:50.298205 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e07f80_e221_4b96_a6ff_03d15f3700d2.slice/crio-a5ce47543a7b1c6e80cf3f1bcf15a4391c9bcb06f90b9d78715adfa267eb0c80 WatchSource:0}: Error finding container a5ce47543a7b1c6e80cf3f1bcf15a4391c9bcb06f90b9d78715adfa267eb0c80: Status 404 returned error can't find the container with id a5ce47543a7b1c6e80cf3f1bcf15a4391c9bcb06f90b9d78715adfa267eb0c80 Jul 08 09:26:50.300595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.300568 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv"] Jul 08 09:26:50.327590 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:50.327551 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" event={"ID":"13e07f80-e221-4b96-a6ff-03d15f3700d2","Type":"ContainerStarted","Data":"a5ce47543a7b1c6e80cf3f1bcf15a4391c9bcb06f90b9d78715adfa267eb0c80"} Jul 08 09:26:51.334600 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:51.334567 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" event={"ID":"13e07f80-e221-4b96-a6ff-03d15f3700d2","Type":"ContainerStarted","Data":"47521c52b158f3fdbe6c1faaa57b9eaed3c5b093ceff42f031cbd9a1ad43e3e2"} Jul 08 09:26:51.335007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:51.334634 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:26:51.356452 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:26:51.356400 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" podStartSLOduration=2.356381204 podStartE2EDuration="2.356381204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:26:51.355383619 +0000 UTC m=+1189.710395207" watchObservedRunningTime="2026-07-08 09:26:51.356381204 +0000 UTC m=+1189.711392770" Jul 08 09:27:02.342171 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:27:02.342143 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rrtqv" Jul 08 09:27:59.869685 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:27:59.869649 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:27:59.936592 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:27:59.936562 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:30:00.159048 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.159009 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29725050-g2jsn"] Jul 08 09:30:00.162722 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.162704 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:00.165532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.165512 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-479jv\"" Jul 08 09:30:00.179532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.179498 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29725050-g2jsn"] Jul 08 09:30:00.224519 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.224487 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsnmq\" (UniqueName: \"kubernetes.io/projected/4a67b618-74cb-43eb-ac6e-12f46709425c-kube-api-access-dsnmq\") pod \"maas-api-key-cleanup-29725050-g2jsn\" (UID: \"4a67b618-74cb-43eb-ac6e-12f46709425c\") " pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:00.325106 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.325070 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsnmq\" (UniqueName: \"kubernetes.io/projected/4a67b618-74cb-43eb-ac6e-12f46709425c-kube-api-access-dsnmq\") pod \"maas-api-key-cleanup-29725050-g2jsn\" (UID: \"4a67b618-74cb-43eb-ac6e-12f46709425c\") " pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:00.335315 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.335279 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsnmq\" (UniqueName: \"kubernetes.io/projected/4a67b618-74cb-43eb-ac6e-12f46709425c-kube-api-access-dsnmq\") pod \"maas-api-key-cleanup-29725050-g2jsn\" (UID: \"4a67b618-74cb-43eb-ac6e-12f46709425c\") " pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:00.474008 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.473926 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:00.613291 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.613261 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29725050-g2jsn"] Jul 08 09:30:00.615228 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:30:00.615198 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a67b618_74cb_43eb_ac6e_12f46709425c.slice/crio-41d4182f5a75c52c8a7924d7c9de6f2da70bb4d8fc392c5f825e30c4139216fe WatchSource:0}: Error finding container 41d4182f5a75c52c8a7924d7c9de6f2da70bb4d8fc392c5f825e30c4139216fe: Status 404 returned error can't find the container with id 41d4182f5a75c52c8a7924d7c9de6f2da70bb4d8fc392c5f825e30c4139216fe Jul 08 09:30:00.617144 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:00.617123 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:30:01.367625 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:01.367586 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" event={"ID":"4a67b618-74cb-43eb-ac6e-12f46709425c","Type":"ContainerStarted","Data":"41d4182f5a75c52c8a7924d7c9de6f2da70bb4d8fc392c5f825e30c4139216fe"} Jul 08 09:30:03.379622 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:03.379580 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" event={"ID":"4a67b618-74cb-43eb-ac6e-12f46709425c","Type":"ContainerStarted","Data":"e46afec8f139b25a87c51b8414b8ce3728d8a921bd8ddcc4f9eec9e5bd894aa6"} Jul 08 09:30:03.400384 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:03.400335 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" podStartSLOduration=0.987597853 podStartE2EDuration="3.400316522s" totalImagesPullingTime="2.412718669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:30:03.399955079 +0000 UTC m=+1381.754966643" watchObservedRunningTime="2026-07-08 09:30:03.400316522 +0000 UTC m=+1381.755328083" Jul 08 09:30:04.385997 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:04.385964 2572 generic.go:356] "Generic (PLEG): container finished" podID="4a67b618-74cb-43eb-ac6e-12f46709425c" containerID="e46afec8f139b25a87c51b8414b8ce3728d8a921bd8ddcc4f9eec9e5bd894aa6" exitCode=0 Jul 08 09:30:04.386408 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:04.386059 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" event={"ID":"4a67b618-74cb-43eb-ac6e-12f46709425c","Type":"ContainerDied","Data":"e46afec8f139b25a87c51b8414b8ce3728d8a921bd8ddcc4f9eec9e5bd894aa6"} Jul 08 09:30:05.528103 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:05.528073 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:05.680083 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:05.679987 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsnmq\" (UniqueName: \"kubernetes.io/projected/4a67b618-74cb-43eb-ac6e-12f46709425c-kube-api-access-dsnmq\") pod \"4a67b618-74cb-43eb-ac6e-12f46709425c\" (UID: \"4a67b618-74cb-43eb-ac6e-12f46709425c\") " Jul 08 09:30:05.682116 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:05.682087 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a67b618-74cb-43eb-ac6e-12f46709425c-kube-api-access-dsnmq" (OuterVolumeSpecName: "kube-api-access-dsnmq") pod "4a67b618-74cb-43eb-ac6e-12f46709425c" (UID: "4a67b618-74cb-43eb-ac6e-12f46709425c"). InnerVolumeSpecName "kube-api-access-dsnmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:30:05.781566 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:05.781529 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsnmq\" (UniqueName: \"kubernetes.io/projected/4a67b618-74cb-43eb-ac6e-12f46709425c-kube-api-access-dsnmq\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:30:06.399936 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:06.399889 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" Jul 08 09:30:06.400111 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:06.399895 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725050-g2jsn" event={"ID":"4a67b618-74cb-43eb-ac6e-12f46709425c","Type":"ContainerDied","Data":"41d4182f5a75c52c8a7924d7c9de6f2da70bb4d8fc392c5f825e30c4139216fe"} Jul 08 09:30:06.400111 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:06.400025 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41d4182f5a75c52c8a7924d7c9de6f2da70bb4d8fc392c5f825e30c4139216fe" Jul 08 09:30:06.433560 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:30:06.433455 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:41.999890 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:41.999806 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:49.271325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:49.271275 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:49.366226 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:49.366190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:49.779452 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:49.779416 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:52.634402 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:52.634365 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:56.511720 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.511684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:56.665371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.665337 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:56.773732 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.773651 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g"] Jul 08 09:32:56.774184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.774169 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a67b618-74cb-43eb-ac6e-12f46709425c" containerName="cleanup" Jul 08 09:32:56.774184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.774185 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a67b618-74cb-43eb-ac6e-12f46709425c" containerName="cleanup" Jul 08 09:32:56.774328 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.774258 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a67b618-74cb-43eb-ac6e-12f46709425c" containerName="cleanup" Jul 08 09:32:56.777566 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.777546 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.780246 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.780226 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8294e149-openshift-default-dockercfg-7qz6t\"" Jul 08 09:32:56.816129 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.816101 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:56.874685 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.874652 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g"] Jul 08 09:32:56.883485 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883457 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-token\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883489 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-podinfo\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883513 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-certs\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883660 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883645 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-socket\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883682 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-data\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883766 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-credential-socket\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883912 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883812 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-envoy\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883912 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883840 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq7bw\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-kube-api-access-vq7bw\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.883912 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.883885 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istiod-ca-cert\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.941216 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.941169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:56.984385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-socket\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984385 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984390 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-data\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984441 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-credential-socket\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984474 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-envoy\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984504 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq7bw\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-kube-api-access-vq7bw\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984552 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istiod-ca-cert\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984592 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-token\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984616 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-podinfo\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.984646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984647 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-certs\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.985050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984822 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-socket\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.985050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984851 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-credential-socket\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.985050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.984978 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-data\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.985215 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.985195 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-certs\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.985332 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.985310 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istiod-ca-cert\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.987183 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.987160 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-envoy\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.987391 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.987369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-podinfo\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:56.999668 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:56.999640 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq7bw\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-kube-api-access-vq7bw\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:57.000718 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.000689 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-token\") pod \"e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:57.089507 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.089438 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:57.261754 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.261728 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g"] Jul 08 09:32:57.264539 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:32:57.264514 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920bf952_8dcb_4988_93d9_4a0bc98cba6a.slice/crio-e3daad2bb742856538625976ec47dffb9c3957b8af08f4b751ed75654a147eca WatchSource:0}: Error finding container e3daad2bb742856538625976ec47dffb9c3957b8af08f4b751ed75654a147eca: Status 404 returned error can't find the container with id e3daad2bb742856538625976ec47dffb9c3957b8af08f4b751ed75654a147eca Jul 08 09:32:57.266675 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.266645 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:32:57.266759 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.266708 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:32:57.266759 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.266736 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:32:57.357340 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.354998 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" event={"ID":"920bf952-8dcb-4988-93d9-4a0bc98cba6a","Type":"ContainerStarted","Data":"e4a8ec0aa8335dc88379c3e6776008e3d6d9bda12379b3dae724ecd4a2391264"} Jul 08 09:32:57.357340 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.356372 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" event={"ID":"920bf952-8dcb-4988-93d9-4a0bc98cba6a","Type":"ContainerStarted","Data":"e3daad2bb742856538625976ec47dffb9c3957b8af08f4b751ed75654a147eca"} Jul 08 09:32:57.386482 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:57.386434 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" podStartSLOduration=1.386418948 podStartE2EDuration="1.386418948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:32:57.386373161 +0000 UTC m=+1555.741384723" watchObservedRunningTime="2026-07-08 09:32:57.386418948 +0000 UTC m=+1555.741430514" Jul 08 09:32:58.072190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.072148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:58.089897 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.089872 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:58.094768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.094745 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:58.224345 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.224304 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:58.360964 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.360933 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:58.361568 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.361545 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:58.362398 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.362379 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:32:58.651925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:58.651823 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:59.541915 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:59.541878 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:32:59.674981 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:32:59.674942 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:01.157085 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:01.157046 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:03.133977 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:03.133936 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:03.280229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:03.280190 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g"] Jul 08 09:33:03.280568 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:03.280422 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" podUID="920bf952-8dcb-4988-93d9-4a0bc98cba6a" containerName="istio-proxy" containerID="cri-o://e4a8ec0aa8335dc88379c3e6776008e3d6d9bda12379b3dae724ecd4a2391264" gracePeriod=30 Jul 08 09:33:03.394105 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:03.393989 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:03.530545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:03.530487 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:03.921760 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:03.921727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:04.765998 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:04.765955 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:08.416281 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.416245 2572 generic.go:356] "Generic (PLEG): container finished" podID="920bf952-8dcb-4988-93d9-4a0bc98cba6a" containerID="e4a8ec0aa8335dc88379c3e6776008e3d6d9bda12379b3dae724ecd4a2391264" exitCode=0 Jul 08 09:33:08.416765 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.416333 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" event={"ID":"920bf952-8dcb-4988-93d9-4a0bc98cba6a","Type":"ContainerDied","Data":"e4a8ec0aa8335dc88379c3e6776008e3d6d9bda12379b3dae724ecd4a2391264"} Jul 08 09:33:08.538882 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.538813 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:33:08.604460 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604422 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-credential-socket\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604488 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istiod-ca-cert\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604533 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-data\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604569 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-envoy\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604613 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-podinfo\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604653 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-certs\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604719 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:08.604979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604729 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-token\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604833 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq7bw\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-kube-api-access-vq7bw\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604930 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-socket\") pod \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\" (UID: \"920bf952-8dcb-4988-93d9-4a0bc98cba6a\") " Jul 08 09:33:08.604979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604955 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-data" (OuterVolumeSpecName: "istio-data") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:08.604979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.604968 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:33:08.605277 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.605247 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:08.605354 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.605331 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:08.605402 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.605365 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istiod-ca-cert\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.605402 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.605383 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-data\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.605402 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.605394 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-socket\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.605512 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.605405 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-credential-socket\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.607032 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.607004 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:08.607139 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.607053 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-kube-api-access-vq7bw" (OuterVolumeSpecName: "kube-api-access-vq7bw") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "kube-api-access-vq7bw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:08.607272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.607245 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:08.607272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.607255 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "920bf952-8dcb-4988-93d9-4a0bc98cba6a" (UID: "920bf952-8dcb-4988-93d9-4a0bc98cba6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:33:08.706943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.706888 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-envoy\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.706943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.706937 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-podinfo\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.706943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.706946 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/920bf952-8dcb-4988-93d9-4a0bc98cba6a-workload-certs\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.706943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.706955 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-istio-token\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:08.707204 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:08.706963 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq7bw\" (UniqueName: \"kubernetes.io/projected/920bf952-8dcb-4988-93d9-4a0bc98cba6a-kube-api-access-vq7bw\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:09.424015 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.423978 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" event={"ID":"920bf952-8dcb-4988-93d9-4a0bc98cba6a","Type":"ContainerDied","Data":"e3daad2bb742856538625976ec47dffb9c3957b8af08f4b751ed75654a147eca"} Jul 08 09:33:09.424015 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.424014 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g" Jul 08 09:33:09.424486 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.424056 2572 scope.go:117] "RemoveContainer" containerID="e4a8ec0aa8335dc88379c3e6776008e3d6d9bda12379b3dae724ecd4a2391264" Jul 08 09:33:09.452010 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.451982 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g"] Jul 08 09:33:09.454229 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.454204 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8294e149-openshift-default-84999d6f55-fhw7g"] Jul 08 09:33:09.668249 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.668209 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:09.769962 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.769847 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:09.917034 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.916997 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp"] Jul 08 09:33:09.917477 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.917463 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="920bf952-8dcb-4988-93d9-4a0bc98cba6a" containerName="istio-proxy" Jul 08 09:33:09.917521 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.917479 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="920bf952-8dcb-4988-93d9-4a0bc98cba6a" containerName="istio-proxy" Jul 08 09:33:09.917591 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.917580 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="920bf952-8dcb-4988-93d9-4a0bc98cba6a" containerName="istio-proxy" Jul 08 09:33:09.922034 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.921809 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:09.925414 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:33:09.925385 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" is forbidden: User \"system:node:ip-10-0-140-252.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-252.ec2.internal' and this object" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:09.925967 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.925950 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-492b5fbf-openshift-default-dockercfg-mjwvx\"" Jul 08 09:33:09.967793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:09.967766 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp"] Jul 08 09:33:10.023455 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023327 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-credential-socket\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023455 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023436 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-socket\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023472 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-token\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-certs\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023559 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-data\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023622 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm2bw\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-kube-api-access-lm2bw\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023661 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-envoy\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023691 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-podinfo\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.023734 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.023725 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istiod-ca-cert\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.082800 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.082762 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:10.125260 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-credential-socket\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125274 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-socket\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125301 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-token\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125339 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-certs\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125370 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-data\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125426 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125419 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm2bw\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-kube-api-access-lm2bw\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125731 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125457 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-envoy\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125731 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125644 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-podinfo\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125731 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125689 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-socket\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125731 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125701 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istiod-ca-cert\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125796 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-credential-socket\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.125929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.125880 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-data\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.126258 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.126217 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-certs\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.126465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.126439 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istiod-ca-cert\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.128110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.128085 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-envoy\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.128296 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.128276 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-podinfo\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.153273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.153236 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm2bw\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-kube-api-access-lm2bw\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.155151 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.155131 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-token\") pod \"e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.177664 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.177635 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="920bf952-8dcb-4988-93d9-4a0bc98cba6a" path="/var/lib/kubelet/pods/920bf952-8dcb-4988-93d9-4a0bc98cba6a/volumes" Jul 08 09:33:10.232329 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.232296 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:10.235222 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.235195 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:10.417094 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.417069 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp"] Jul 08 09:33:10.419009 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:33:10.418981 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea2464f_b6e8_4ff8_b533_9ca91ae76311.slice/crio-fefb0d4a9690bcf0bb4cbfffbd215015665c64091cdfa7ee4fb9e3af0c403367 WatchSource:0}: Error finding container fefb0d4a9690bcf0bb4cbfffbd215015665c64091cdfa7ee4fb9e3af0c403367: Status 404 returned error can't find the container with id fefb0d4a9690bcf0bb4cbfffbd215015665c64091cdfa7ee4fb9e3af0c403367 Jul 08 09:33:10.421278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.421249 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:10.421358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.421310 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:10.421358 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.421340 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:10.431467 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:10.431441 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" event={"ID":"6ea2464f-b6e8-4ff8-b533-9ca91ae76311","Type":"ContainerStarted","Data":"fefb0d4a9690bcf0bb4cbfffbd215015665c64091cdfa7ee4fb9e3af0c403367"} Jul 08 09:33:11.209431 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:11.209401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:11.440868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:11.440829 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" event={"ID":"6ea2464f-b6e8-4ff8-b533-9ca91ae76311","Type":"ContainerStarted","Data":"0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d"} Jul 08 09:33:11.441806 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:11.441777 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:11.485733 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:11.485632 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" podStartSLOduration=2.485613605 podStartE2EDuration="2.485613605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:33:11.484250234 +0000 UTC m=+1569.839261798" watchObservedRunningTime="2026-07-08 09:33:11.485613605 +0000 UTC m=+1569.840625166" Jul 08 09:33:11.591771 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:11.591735 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:11.823969 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:11.823875 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:12.235694 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:12.235659 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:12.240205 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:12.240182 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:12.329747 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:12.329699 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:12.446955 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:12.446924 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:12.448048 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:12.448027 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:12.471530 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:12.471500 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:13.361222 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:13.361193 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:16.374288 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:16.374253 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:17.030453 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:17.030415 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:22.504014 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.503981 2572 generic.go:356] "Generic (PLEG): container finished" podID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerID="0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d" exitCode=0 Jul 08 09:33:22.504430 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.504054 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" event={"ID":"6ea2464f-b6e8-4ff8-b533-9ca91ae76311","Type":"ContainerDied","Data":"0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d"} Jul 08 09:33:22.504506 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.504490 2572 scope.go:117] "RemoveContainer" containerID="0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d" Jul 08 09:33:22.505714 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.505679 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:22.505825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.505758 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:22.505825 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.505807 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:33:22.548770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.548743 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp"] Jul 08 09:33:22.629925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.629556 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:22.760109 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:22.760014 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:23.512006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:23.511960 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" event={"ID":"6ea2464f-b6e8-4ff8-b533-9ca91ae76311","Type":"ContainerStarted","Data":"d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e"} Jul 08 09:33:23.512467 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:23.512031 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" containerID="cri-o://d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e" gracePeriod=30 Jul 08 09:33:27.447123 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:27.447092 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:28.779489 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.779421 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:28.805689 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.805661 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-token\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.805854 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.805695 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-envoy\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.805854 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.805730 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-podinfo\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.805848 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-data\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.805947 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-credential-socket\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.805974 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-socket\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806028 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-certs\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806103 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm2bw\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-kube-api-access-lm2bw\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806145 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istiod-ca-cert\") pod \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\" (UID: \"6ea2464f-b6e8-4ff8-b533-9ca91ae76311\") " Jul 08 09:33:28.806337 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806208 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:28.806429 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806380 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:28.806519 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806359 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:28.806647 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806627 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-credential-socket\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.806711 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806655 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-socket\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.806768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806718 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-workload-certs\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.806768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806688 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:33:28.806768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.806708 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:28.807981 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.807945 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:28.808121 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.808101 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:33:28.808290 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.808268 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-kube-api-access-lm2bw" (OuterVolumeSpecName: "kube-api-access-lm2bw") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "kube-api-access-lm2bw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:33:28.808652 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.808633 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ea2464f-b6e8-4ff8-b533-9ca91ae76311" (UID: "6ea2464f-b6e8-4ff8-b533-9ca91ae76311"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:33:28.907824 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.907789 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm2bw\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-kube-api-access-lm2bw\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.907824 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.907817 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istiod-ca-cert\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.907824 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.907828 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-token\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.908094 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.907838 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-envoy\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.908094 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.907846 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-podinfo\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:28.908094 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:28.907853 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ea2464f-b6e8-4ff8-b533-9ca91ae76311-istio-data\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:33:29.554495 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.554453 2572 generic.go:356] "Generic (PLEG): container finished" podID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerID="d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e" exitCode=0 Jul 08 09:33:29.554654 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.554525 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" Jul 08 09:33:29.554654 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.554546 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" event={"ID":"6ea2464f-b6e8-4ff8-b533-9ca91ae76311","Type":"ContainerDied","Data":"d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e"} Jul 08 09:33:29.554654 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.554625 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp" event={"ID":"6ea2464f-b6e8-4ff8-b533-9ca91ae76311","Type":"ContainerDied","Data":"fefb0d4a9690bcf0bb4cbfffbd215015665c64091cdfa7ee4fb9e3af0c403367"} Jul 08 09:33:29.554654 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.554648 2572 scope.go:117] "RemoveContainer" containerID="d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e" Jul 08 09:33:29.564855 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.564837 2572 scope.go:117] "RemoveContainer" containerID="0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d" Jul 08 09:33:29.573096 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.573079 2572 scope.go:117] "RemoveContainer" containerID="d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e" Jul 08 09:33:29.573324 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:33:29.573307 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e\": container with ID starting with d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e not found: ID does not exist" containerID="d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e" Jul 08 09:33:29.573373 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.573331 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e"} err="failed to get container status \"d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e\": rpc error: code = NotFound desc = could not find container \"d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e\": container with ID starting with d91e7f8ed3d27812ca7de0d12d480e22152acc21b72d790494ce1fd2b3b13a0e not found: ID does not exist" Jul 08 09:33:29.573373 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.573347 2572 scope.go:117] "RemoveContainer" containerID="0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d" Jul 08 09:33:29.573607 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:33:29.573589 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d\": container with ID starting with 0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d not found: ID does not exist" containerID="0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d" Jul 08 09:33:29.573656 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.573612 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d"} err="failed to get container status \"0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d\": rpc error: code = NotFound desc = could not find container \"0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d\": container with ID starting with 0d24548eb272099276251c77786536b69409feb5432f2761d24d45ef918a820d not found: ID does not exist" Jul 08 09:33:29.581405 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.581379 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp"] Jul 08 09:33:29.585190 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.585170 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-492b5fbf-openshift-default-b8f4dd6c5-twvqp"] Jul 08 09:33:29.594603 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.594573 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:29.671406 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.671379 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:29.927979 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:29.927944 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:30.033054 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:30.033019 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:30.176873 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:30.176844 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" path="/var/lib/kubelet/pods/6ea2464f-b6e8-4ff8-b533-9ca91ae76311/volumes" Jul 08 09:33:31.100705 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:31.100655 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:31.281916 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:31.281866 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:31.421123 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:31.421091 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:31.893314 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:31.893284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:32.035896 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:32.035861 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:32.172361 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:32.172265 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:32.954224 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:32.954172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:35.475371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:35.475333 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:35.680980 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:35.680945 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:35.836769 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:35.836679 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:36.029090 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:36.029057 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:38.058874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:38.058834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:40.368276 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:40.368239 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:42.032786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:42.032744 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:42.140709 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:42.140668 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:42.553849 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:42.553820 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:42.663000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:42.662969 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:42.781772 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:42.781740 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:44.379464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:44.379430 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:50.047688 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:50.047650 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:50.156611 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:50.156561 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:50.280303 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:50.280270 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:50.383441 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:50.383396 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:33:50.848330 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:33:50.848220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:16.896104 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:16.896062 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:23.526462 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:23.526423 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:23.678886 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:23.678848 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:23.846343 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:23.846220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:39.794463 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:39.794428 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:46.344732 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:46.344697 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:46.456211 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:46.456171 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:46.555780 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:46.555751 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:53.853548 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:53.853512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:57.182851 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:57.182809 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:34:58.029256 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:34:58.029220 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:04.302068 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:04.302032 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:04.434169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:04.434133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:04.561861 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:04.561761 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:05.386843 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:05.386809 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:05.726408 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:05.726314 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:07.329831 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:07.329786 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:07.423984 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:07.423949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:13.403327 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:13.403289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:14.846275 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:14.846217 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:20.481259 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:20.481225 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:20.651177 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:20.651136 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:20.850258 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:20.850162 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:21.175247 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:21.175210 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:21.336579 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:21.336543 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:24.185086 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:24.184992 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:24.316573 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:24.316537 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:25.039062 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:25.039021 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:25.144582 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:25.144545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:25.238101 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:25.238067 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:28.702207 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:28.702170 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:29.461955 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:29.461891 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:29.656147 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:29.656103 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:29.877776 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:29.877745 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:36.097882 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:36.097834 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:36.197169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:36.197137 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:36.540824 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:36.540783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:36.975735 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:36.975699 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:37.056053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:37.056016 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:43.498040 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:43.497998 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:43.607233 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:43.607175 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:43.799612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:43.799512 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:43.999620 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:43.999560 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:44.125208 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:44.125172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:44.284449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:44.284415 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:44.380684 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:44.380590 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:50.454827 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:50.454783 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:50.800527 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:50.800439 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:51.116956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:51.116893 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:56.933509 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:56.933468 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:35:57.020698 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:35:57.020656 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:03.888539 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:03.888502 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:04.224927 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:04.224809 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:04.461598 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:04.461554 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:06.932743 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:06.932705 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:07.028213 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:07.028148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:07.184218 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:07.184133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:07.288695 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:07.288646 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:07.709932 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:07.709867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:07.848757 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:07.848717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:11.450401 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:11.450365 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:20.819412 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:20.819372 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:20.921481 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:20.921445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:26.191181 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:26.191152 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:29.053740 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:29.053696 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:29.276470 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:29.276433 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:54.363559 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:54.363465 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:36:55.055761 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:36:55.055729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:00.513164 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:00.513127 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:05.549399 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:05.549352 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:41.895078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:41.895020 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:43.204262 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:43.204224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:49.078799 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:49.078746 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:49.147932 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:49.147867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:49.493640 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:49.493602 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:49.682987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:49.682894 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:50.863423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:50.863389 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:50.967301 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:50.967243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:51.801459 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:51.801416 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:51.891696 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:51.891642 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:51.994611 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:51.994579 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:53.456279 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:53.456243 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:59.781000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:59.780958 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:37:59.861236 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:37:59.861205 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:00.511250 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:00.511205 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:06.304952 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:06.304895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:06.378029 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:06.377983 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:06.482992 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:06.482957 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:06.579090 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:06.579009 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:14.586744 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:14.586702 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:17.133430 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:17.133389 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:17.468228 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:17.468142 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:38.489498 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:38.489457 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:44.683938 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:44.683880 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:44.754083 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:44.754048 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:45.069731 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:45.069628 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:45.438847 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:45.438817 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:45.506773 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:45.506731 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:48.875390 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:48.875354 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:49.000720 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:49.000681 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:49.421296 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:49.421240 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:49.530647 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:49.530606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:49.718770 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:49.718657 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:53.297523 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.297493 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:53.613178 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.613147 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:53.902534 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.902371 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz"] Jul 08 09:38:53.903171 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.903143 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" Jul 08 09:38:53.903171 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.903169 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" Jul 08 09:38:53.903363 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.903200 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" Jul 08 09:38:53.903363 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.903210 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" Jul 08 09:38:53.903363 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.903324 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" Jul 08 09:38:53.906571 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.906550 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.909382 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.909361 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-802008-openshift-default-dockercfg-8kjw6\"" Jul 08 09:38:53.919117 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.919092 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz"] Jul 08 09:38:53.941574 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.941547 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:53.993847 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.993817 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9778add-7369-4696-b35b-8632be364512-istiod-ca-cert\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.993847 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.993850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-socket\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.993879 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htgxf\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-kube-api-access-htgxf\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.993962 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-data\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994120 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.994062 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-envoy\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994120 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.994109 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-credential-socket\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994198 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.994169 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-istio-token\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994198 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.994188 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-certs\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:53.994272 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:53.994216 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9778add-7369-4696-b35b-8632be364512-istio-podinfo\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.041110 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.041084 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:54.095352 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095322 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-envoy\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095526 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095369 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-credential-socket\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095526 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095501 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-istio-token\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095632 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095535 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-certs\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095632 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095568 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9778add-7369-4696-b35b-8632be364512-istio-podinfo\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095632 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095617 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9778add-7369-4696-b35b-8632be364512-istiod-ca-cert\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-socket\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095694 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htgxf\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-kube-api-access-htgxf\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095731 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-data\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095803 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-credential-socket\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.095987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.095950 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-certs\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.096336 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.096309 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-socket\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.096435 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.096388 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-data\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.096794 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.096769 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9778add-7369-4696-b35b-8632be364512-istiod-ca-cert\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.098423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.098405 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-envoy\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.098709 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.098687 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9778add-7369-4696-b35b-8632be364512-istio-podinfo\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.105609 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.105588 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htgxf\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-kube-api-access-htgxf\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.106286 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.106264 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-istio-token\") pod \"e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.217318 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.217230 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:54.362219 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.362195 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz"] Jul 08 09:38:54.364607 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:38:54.364574 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9778add_7369_4696_b35b_8632be364512.slice/crio-ec95004f3d4f2bf17237294cb65fbc15800d50b987d50511f035fbfa707b1e56 WatchSource:0}: Error finding container ec95004f3d4f2bf17237294cb65fbc15800d50b987d50511f035fbfa707b1e56: Status 404 returned error can't find the container with id ec95004f3d4f2bf17237294cb65fbc15800d50b987d50511f035fbfa707b1e56 Jul 08 09:38:54.366696 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.366662 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:38:54.366784 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.366726 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:38:54.366784 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:54.366759 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:38:55.326050 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:55.326009 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" event={"ID":"b9778add-7369-4696-b35b-8632be364512","Type":"ContainerStarted","Data":"acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86"} Jul 08 09:38:55.326268 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:55.326080 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" event={"ID":"b9778add-7369-4696-b35b-8632be364512","Type":"ContainerStarted","Data":"ec95004f3d4f2bf17237294cb65fbc15800d50b987d50511f035fbfa707b1e56"} Jul 08 09:38:55.349080 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:55.349028 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" podStartSLOduration=2.349009285 podStartE2EDuration="2.349009285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:38:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:38:55.348640783 +0000 UTC m=+1913.703652378" watchObservedRunningTime="2026-07-08 09:38:55.349009285 +0000 UTC m=+1913.704020848" Jul 08 09:38:56.217708 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:56.217672 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:56.222543 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:56.222515 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:56.331970 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:56.331939 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:56.333023 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:56.333002 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:38:57.352719 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:57.352667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:57.469181 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:57.469150 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:57.566815 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:57.566782 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:58.029283 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:58.029251 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:58.134304 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:58.134272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:38:58.261074 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:38:58.261033 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:01.896744 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:01.896709 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:10.411473 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.411442 2572 generic.go:356] "Generic (PLEG): container finished" podID="b9778add-7369-4696-b35b-8632be364512" containerID="acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86" exitCode=0 Jul 08 09:39:10.412006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.411516 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" event={"ID":"b9778add-7369-4696-b35b-8632be364512","Type":"ContainerDied","Data":"acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86"} Jul 08 09:39:10.412006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.411977 2572 scope.go:117] "RemoveContainer" containerID="acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86" Jul 08 09:39:10.413119 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.413087 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:39:10.413193 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.413161 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:39:10.413233 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.413193 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 08 09:39:10.609177 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.609131 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:10.730547 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:10.730509 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:11.332208 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:11.332170 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:39:11.418928 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:11.418882 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" event={"ID":"b9778add-7369-4696-b35b-8632be364512","Type":"ContainerStarted","Data":"551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084"} Jul 08 09:39:12.218360 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:12.218328 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:39:12.219517 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:12.219495 2572 patch_prober.go:28] interesting pod/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 09:39:12.219630 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:12.219563 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 09:39:13.218597 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:13.218556 2572 patch_prober.go:28] interesting pod/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 09:39:13.219053 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:13.218625 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 09:39:14.217733 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:14.217701 2572 patch_prober.go:28] interesting pod/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 08 09:39:14.217986 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:14.217754 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 08 09:39:14.755959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:14.755878 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:15.237790 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:15.237709 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:39:15.238297 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:15.238271 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:39:15.239074 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:15.239055 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:39:18.963511 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:18.963472 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:19.065344 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:19.065303 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:39:49.451478 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:39:49.451444 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:11.053978 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:11.053937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:11.839757 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:11.839718 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:12.027497 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:12.027457 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:12.470393 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:12.470359 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:12.619507 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:12.619460 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:12.762776 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:12.762684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:12.926721 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:12.926687 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:17.527085 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:17.527043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:54.674458 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:54.674418 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:54.970943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:54.970822 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:55.205029 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:55.204989 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:55.518179 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:55.518141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:55.910137 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:55.910102 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:56.194712 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:56.194617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:56.301371 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:56.301341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:40:56.613200 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:40:56.613167 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:02.019131 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.019098 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:02.146287 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.146247 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:02.280169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.280087 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz"] Jul 08 09:41:02.280331 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.280293 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" containerID="cri-o://551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084" gracePeriod=30 Jul 08 09:41:02.355162 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.355122 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:02.570134 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.570042 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:02.926617 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:02.926577 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:03.290998 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:03.290911 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:03.538252 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:03.538208 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:06.389038 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:06.389003 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:06.487724 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:06.487688 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:06.593099 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:06.593063 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:06.848278 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:06.848199 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:06.983042 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:06.983008 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:07.085491 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.085461 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:07.547122 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.547061 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:41:07.651941 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.651885 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-certs\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.651954 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-credential-socket\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.651989 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-istio-token\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652022 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-envoy\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652068 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9778add-7369-4696-b35b-8632be364512-istiod-ca-cert\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652087 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htgxf\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-kube-api-access-htgxf\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652128 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652125 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-data\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652145 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:07.652465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652242 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:07.652465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652182 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9778add-7369-4696-b35b-8632be364512-istio-podinfo\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652465 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652318 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-socket\") pod \"b9778add-7369-4696-b35b-8632be364512\" (UID: \"b9778add-7369-4696-b35b-8632be364512\") " Jul 08 09:41:07.652662 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652507 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9778add-7369-4696-b35b-8632be364512-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:41:07.652860 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652840 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-certs\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.652954 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652866 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-credential-socket\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.652954 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652880 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9778add-7369-4696-b35b-8632be364512-istiod-ca-cert\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.652954 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.652916 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:07.653211 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.653182 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:07.654286 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.654262 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:07.654501 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.654488 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9778add-7369-4696-b35b-8632be364512-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 09:41:07.654667 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.654645 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-kube-api-access-htgxf" (OuterVolumeSpecName: "kube-api-access-htgxf") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "kube-api-access-htgxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:07.654787 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.654774 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9778add-7369-4696-b35b-8632be364512" (UID: "b9778add-7369-4696-b35b-8632be364512"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 09:41:07.754377 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.754346 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-envoy\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.754377 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.754374 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htgxf\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-kube-api-access-htgxf\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.754377 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.754384 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-istio-data\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.754612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.754392 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9778add-7369-4696-b35b-8632be364512-istio-podinfo\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.754612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.754400 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9778add-7369-4696-b35b-8632be364512-workload-socket\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:07.754612 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:07.754408 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9778add-7369-4696-b35b-8632be364512-istio-token\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:08.078006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.077968 2572 generic.go:356] "Generic (PLEG): container finished" podID="b9778add-7369-4696-b35b-8632be364512" containerID="551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084" exitCode=0 Jul 08 09:41:08.078185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.078028 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" Jul 08 09:41:08.078185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.078065 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" event={"ID":"b9778add-7369-4696-b35b-8632be364512","Type":"ContainerDied","Data":"551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084"} Jul 08 09:41:08.078185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.078147 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz" event={"ID":"b9778add-7369-4696-b35b-8632be364512","Type":"ContainerDied","Data":"ec95004f3d4f2bf17237294cb65fbc15800d50b987d50511f035fbfa707b1e56"} Jul 08 09:41:08.078185 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.078171 2572 scope.go:117] "RemoveContainer" containerID="551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084" Jul 08 09:41:08.087768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.087748 2572 scope.go:117] "RemoveContainer" containerID="acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86" Jul 08 09:41:08.096259 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.096238 2572 scope.go:117] "RemoveContainer" containerID="551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084" Jul 08 09:41:08.096542 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:41:08.096519 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084\": container with ID starting with 551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084 not found: ID does not exist" containerID="551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084" Jul 08 09:41:08.096617 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.096557 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084"} err="failed to get container status \"551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084\": rpc error: code = NotFound desc = could not find container \"551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084\": container with ID starting with 551a0b8e6bc7bebeeb5d5dcdfa7a0cecc0bb0575bff9b24045ba4d2d930b1084 not found: ID does not exist" Jul 08 09:41:08.096617 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.096584 2572 scope.go:117] "RemoveContainer" containerID="acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86" Jul 08 09:41:08.096877 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:41:08.096860 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86\": container with ID starting with acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86 not found: ID does not exist" containerID="acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86" Jul 08 09:41:08.096940 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.096883 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86"} err="failed to get container status \"acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86\": rpc error: code = NotFound desc = could not find container \"acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86\": container with ID starting with acea8e27cf1ea50eaf111f3ee18a05fda2f68fbec89efc2f96f23fd6884f5e86 not found: ID does not exist" Jul 08 09:41:08.141505 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.141473 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz"] Jul 08 09:41:08.148712 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.148684 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-802008-openshift-default-77b45c8f67-4f9bz"] Jul 08 09:41:08.176680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:08.176639 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9778add-7369-4696-b35b-8632be364512" path="/var/lib/kubelet/pods/b9778add-7369-4696-b35b-8632be364512/volumes" Jul 08 09:41:10.914969 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:10.914934 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:11.424379 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:11.424345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:11.771852 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:11.771767 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:12.102736 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:12.102647 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:14.424146 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.424109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:14.544623 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.544588 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd"] Jul 08 09:41:14.545356 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545322 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" Jul 08 09:41:14.545356 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545348 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" Jul 08 09:41:14.545543 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545389 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" Jul 08 09:41:14.545543 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545397 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" Jul 08 09:41:14.545543 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545510 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea2464f-b6e8-4ff8-b533-9ca91ae76311" containerName="istio-proxy" Jul 08 09:41:14.545543 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545526 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" Jul 08 09:41:14.545543 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.545538 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9778add-7369-4696-b35b-8632be364512" containerName="istio-proxy" Jul 08 09:41:14.550779 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.550761 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.554168 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.554151 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-4368cb-serving-cert\"" Jul 08 09:41:14.554268 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.554153 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-479jv\"" Jul 08 09:41:14.566055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.566033 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd"] Jul 08 09:41:14.568169 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.568133 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:14.651614 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.651575 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:14.730273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.730184 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0722af3-15de-4134-a0e3-b6dfc708ea6e-openshift-service-ca\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.730273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.730229 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0722af3-15de-4134-a0e3-b6dfc708ea6e-maas-api-tls\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.730517 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.730357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5k8r\" (UniqueName: \"kubernetes.io/projected/c0722af3-15de-4134-a0e3-b6dfc708ea6e-kube-api-access-l5k8r\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.831786 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.831746 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0722af3-15de-4134-a0e3-b6dfc708ea6e-openshift-service-ca\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.831981 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.831809 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0722af3-15de-4134-a0e3-b6dfc708ea6e-maas-api-tls\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.832040 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.832001 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5k8r\" (UniqueName: \"kubernetes.io/projected/c0722af3-15de-4134-a0e3-b6dfc708ea6e-kube-api-access-l5k8r\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.832614 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.832580 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0722af3-15de-4134-a0e3-b6dfc708ea6e-openshift-service-ca\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.834489 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.834466 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0722af3-15de-4134-a0e3-b6dfc708ea6e-maas-api-tls\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.840510 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.840491 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5k8r\" (UniqueName: \"kubernetes.io/projected/c0722af3-15de-4134-a0e3-b6dfc708ea6e-kube-api-access-l5k8r\") pod \"maas-api-e2e-api-b-4368cb-697588cc45-g5ccd\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:14.861515 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:14.861496 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:15.012983 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:15.012959 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd"] Jul 08 09:41:15.015330 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:41:15.015305 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0722af3_15de_4134_a0e3_b6dfc708ea6e.slice/crio-bc95f040ad62fe664f6644d9f8d0ea0f04560b3732b21a4077e6f6a3e5ef88ec WatchSource:0}: Error finding container bc95f040ad62fe664f6644d9f8d0ea0f04560b3732b21a4077e6f6a3e5ef88ec: Status 404 returned error can't find the container with id bc95f040ad62fe664f6644d9f8d0ea0f04560b3732b21a4077e6f6a3e5ef88ec Jul 08 09:41:15.016768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:15.016750 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 09:41:15.111707 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:15.111667 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:15.128837 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:15.128735 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" event={"ID":"c0722af3-15de-4134-a0e3-b6dfc708ea6e","Type":"ContainerStarted","Data":"bc95f040ad62fe664f6644d9f8d0ea0f04560b3732b21a4077e6f6a3e5ef88ec"} Jul 08 09:41:15.343209 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:15.343111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:16.413327 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:16.413286 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:16.511382 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:16.511345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:17.140678 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:17.140592 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" event={"ID":"c0722af3-15de-4134-a0e3-b6dfc708ea6e","Type":"ContainerStarted","Data":"26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784"} Jul 08 09:41:17.140823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:17.140724 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:17.180000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:17.179951 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" podStartSLOduration=1.340040781 podStartE2EDuration="3.179935887s" totalImagesPullingTime="1.839895106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:17.177979751 +0000 UTC m=+2055.532991314" watchObservedRunningTime="2026-07-08 09:41:17.179935887 +0000 UTC m=+2055.534947448" Jul 08 09:41:18.920102 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:18.920071 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:21.667418 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:21.667380 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:21.766652 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:21.766614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:23.151807 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:23.151780 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:27.194325 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:27.194289 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:27.858348 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:27.858305 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:28.049008 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:28.048971 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:28.151541 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:28.151452 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:34.151432 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:34.151393 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:34.257012 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:34.256979 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:34.459431 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:34.459328 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:34.678326 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:34.678293 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:34.856089 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:34.856052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:35.324007 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.323882 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:35.424545 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.424505 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:35.599566 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.599489 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd"] Jul 08 09:41:35.599749 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.599706 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" podUID="c0722af3-15de-4134-a0e3-b6dfc708ea6e" containerName="maas-api" containerID="cri-o://26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784" gracePeriod=30 Jul 08 09:41:35.638554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.638526 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:35.859520 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.859497 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:35.943727 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.943701 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5k8r\" (UniqueName: \"kubernetes.io/projected/c0722af3-15de-4134-a0e3-b6dfc708ea6e-kube-api-access-l5k8r\") pod \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " Jul 08 09:41:35.943868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.943744 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0722af3-15de-4134-a0e3-b6dfc708ea6e-openshift-service-ca\") pod \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " Jul 08 09:41:35.943868 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.943797 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0722af3-15de-4134-a0e3-b6dfc708ea6e-maas-api-tls\") pod \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\" (UID: \"c0722af3-15de-4134-a0e3-b6dfc708ea6e\") " Jul 08 09:41:35.944189 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.944166 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0722af3-15de-4134-a0e3-b6dfc708ea6e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0722af3-15de-4134-a0e3-b6dfc708ea6e" (UID: "c0722af3-15de-4134-a0e3-b6dfc708ea6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:41:35.944269 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.944254 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0722af3-15de-4134-a0e3-b6dfc708ea6e-openshift-service-ca\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:35.945925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.945877 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0722af3-15de-4134-a0e3-b6dfc708ea6e-kube-api-access-l5k8r" (OuterVolumeSpecName: "kube-api-access-l5k8r") pod "c0722af3-15de-4134-a0e3-b6dfc708ea6e" (UID: "c0722af3-15de-4134-a0e3-b6dfc708ea6e"). InnerVolumeSpecName "kube-api-access-l5k8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:41:35.946004 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:35.945882 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0722af3-15de-4134-a0e3-b6dfc708ea6e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0722af3-15de-4134-a0e3-b6dfc708ea6e" (UID: "c0722af3-15de-4134-a0e3-b6dfc708ea6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:41:36.044793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.044761 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5k8r\" (UniqueName: \"kubernetes.io/projected/c0722af3-15de-4134-a0e3-b6dfc708ea6e-kube-api-access-l5k8r\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:36.044793 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.044786 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0722af3-15de-4134-a0e3-b6dfc708ea6e-maas-api-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:41:36.242961 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.242927 2572 generic.go:356] "Generic (PLEG): container finished" podID="c0722af3-15de-4134-a0e3-b6dfc708ea6e" containerID="26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784" exitCode=0 Jul 08 09:41:36.243156 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.242971 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" Jul 08 09:41:36.243156 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.243026 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" event={"ID":"c0722af3-15de-4134-a0e3-b6dfc708ea6e","Type":"ContainerDied","Data":"26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784"} Jul 08 09:41:36.243156 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.243083 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd" event={"ID":"c0722af3-15de-4134-a0e3-b6dfc708ea6e","Type":"ContainerDied","Data":"bc95f040ad62fe664f6644d9f8d0ea0f04560b3732b21a4077e6f6a3e5ef88ec"} Jul 08 09:41:36.243156 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.243114 2572 scope.go:117] "RemoveContainer" containerID="26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784" Jul 08 09:41:36.252232 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.252215 2572 scope.go:117] "RemoveContainer" containerID="26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784" Jul 08 09:41:36.252489 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:41:36.252470 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784\": container with ID starting with 26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784 not found: ID does not exist" containerID="26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784" Jul 08 09:41:36.252554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.252496 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784"} err="failed to get container status \"26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784\": rpc error: code = NotFound desc = could not find container \"26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784\": container with ID starting with 26f3ad129b1a958018bc9b2b2d1a2856be8c1e3536f10da038ee804d94126784 not found: ID does not exist" Jul 08 09:41:36.262944 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.262919 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd"] Jul 08 09:41:36.267095 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:36.267059 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-4368cb-697588cc45-g5ccd"] Jul 08 09:41:38.175958 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:38.175928 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0722af3-15de-4134-a0e3-b6dfc708ea6e" path="/var/lib/kubelet/pods/c0722af3-15de-4134-a0e3-b6dfc708ea6e/volumes" Jul 08 09:41:41.398381 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:41.398347 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:41.498606 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:41.498567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:41.778374 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:41.778283 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:41.957709 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:41.957673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:42.318404 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:42.318346 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:42.685541 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:42.685501 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:42.785149 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:42.785116 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:45.733943 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.733877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:45.807464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.807435 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw"] Jul 08 09:41:45.808005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.807987 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0722af3-15de-4134-a0e3-b6dfc708ea6e" containerName="maas-api" Jul 08 09:41:45.808005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.808007 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0722af3-15de-4134-a0e3-b6dfc708ea6e" containerName="maas-api" Jul 08 09:41:45.808175 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.808147 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0722af3-15de-4134-a0e3-b6dfc708ea6e" containerName="maas-api" Jul 08 09:41:45.813248 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.813227 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:45.819831 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.819808 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-479jv\"" Jul 08 09:41:45.820055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.820037 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-8c4427-serving-cert\"" Jul 08 09:41:45.830765 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.830744 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw"] Jul 08 09:41:45.842223 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.842179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:45.950041 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.950007 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:45.950223 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.950052 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:45.950223 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.950090 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28772\" (UniqueName: \"kubernetes.io/projected/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-kube-api-access-28772\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:45.983737 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:45.983703 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:46.050661 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.050587 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.050661 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.050626 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.050837 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.050662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28772\" (UniqueName: \"kubernetes.io/projected/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-kube-api-access-28772\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.051343 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.051317 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.053135 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.053115 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.061797 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.061767 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28772\" (UniqueName: \"kubernetes.io/projected/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-kube-api-access-28772\") pod \"maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.118462 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.118426 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:46.124342 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.124312 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:46.275518 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.275479 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw"] Jul 08 09:41:46.276454 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:41:46.276427 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b4fb87_3ee3_4c7c_b306_f65e5c204a20.slice/crio-8f65b5afd1e463e6d9ba9b3a4249c1b590f14d62750b7a33add92461280a12a3 WatchSource:0}: Error finding container 8f65b5afd1e463e6d9ba9b3a4249c1b590f14d62750b7a33add92461280a12a3: Status 404 returned error can't find the container with id 8f65b5afd1e463e6d9ba9b3a4249c1b590f14d62750b7a33add92461280a12a3 Jul 08 09:41:46.300327 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.300296 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" event={"ID":"98b4fb87-3ee3-4c7c-b306-f65e5c204a20","Type":"ContainerStarted","Data":"8f65b5afd1e463e6d9ba9b3a4249c1b590f14d62750b7a33add92461280a12a3"} Jul 08 09:41:46.452788 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.452745 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:46.550400 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:46.550318 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:47.307135 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:47.307097 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" event={"ID":"98b4fb87-3ee3-4c7c-b306-f65e5c204a20","Type":"ContainerStarted","Data":"bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392"} Jul 08 09:41:47.307531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:47.307203 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:47.334567 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:47.334503 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" podStartSLOduration=1.803451193 podStartE2EDuration="2.334484434s" totalImagesPullingTime="531.033241ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:47.33343673 +0000 UTC m=+2085.688448292" watchObservedRunningTime="2026-07-08 09:41:47.334484434 +0000 UTC m=+2085.689495997" Jul 08 09:41:50.162062 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:50.162029 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:50.591805 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:50.591701 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:50.995217 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:50.995172 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:51.336153 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:51.336065 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:53.318832 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.318802 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:41:53.650966 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.650860 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:53.755091 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.755057 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t"] Jul 08 09:41:53.759129 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.759106 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.762372 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.762351 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-f0ff0b-serving-cert\"" Jul 08 09:41:53.772130 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.772097 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t"] Jul 08 09:41:53.785296 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.785264 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:53.828344 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.828322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.828480 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.828380 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrbm\" (UniqueName: \"kubernetes.io/projected/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-kube-api-access-xgrbm\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.828480 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.828468 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.929179 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.929087 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.929319 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.929187 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrbm\" (UniqueName: \"kubernetes.io/projected/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-kube-api-access-xgrbm\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.929319 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.929246 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.929319 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:41:53.929249 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-f0ff0b-serving-cert: secret "maas-api-e2e-shared-b-f0ff0b-serving-cert" not found Jul 08 09:41:53.929446 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:41:53.929320 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls podName:fa7b0996-9f9d-4319-b0f9-f3fae96605cb nodeName:}" failed. No retries permitted until 2026-07-08 09:41:54.429298831 +0000 UTC m=+2092.784310390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls") pod "maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" (UID: "fa7b0996-9f9d-4319-b0f9-f3fae96605cb") : secret "maas-api-e2e-shared-b-f0ff0b-serving-cert" not found Jul 08 09:41:53.930063 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.930042 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:53.939105 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:53.939077 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrbm\" (UniqueName: \"kubernetes.io/projected/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-kube-api-access-xgrbm\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:54.005464 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.005423 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:54.326067 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.325981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:54.430752 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.430711 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:54.434856 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.434831 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:54.437352 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.437331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:54.533005 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.532967 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:41:54.671086 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.671060 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:54.817538 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:54.817515 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t"] Jul 08 09:41:54.819386 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:41:54.819353 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7b0996_9f9d_4319_b0f9_f3fae96605cb.slice/crio-2a3ae82f518a96a99c46b6fc319e5768357f756ba2557f8cb1d88792d63df1a8 WatchSource:0}: Error finding container 2a3ae82f518a96a99c46b6fc319e5768357f756ba2557f8cb1d88792d63df1a8: Status 404 returned error can't find the container with id 2a3ae82f518a96a99c46b6fc319e5768357f756ba2557f8cb1d88792d63df1a8 Jul 08 09:41:55.354324 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:55.354285 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" event={"ID":"fa7b0996-9f9d-4319-b0f9-f3fae96605cb","Type":"ContainerStarted","Data":"2a3ae82f518a96a99c46b6fc319e5768357f756ba2557f8cb1d88792d63df1a8"} Jul 08 09:41:56.361055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:56.361020 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" event={"ID":"fa7b0996-9f9d-4319-b0f9-f3fae96605cb","Type":"ContainerStarted","Data":"c9943f050489f802d48d0f413a61c766febc1cb34254f87ea0cd6fbe067d81c0"} Jul 08 09:41:56.361415 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:56.361152 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:41:56.385701 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:56.385655 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" podStartSLOduration=2.712705825 podStartE2EDuration="3.385642336s" totalImagesPullingTime="672.936511ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:41:56.384267544 +0000 UTC m=+2094.739279106" watchObservedRunningTime="2026-07-08 09:41:56.385642336 +0000 UTC m=+2094.740653922" Jul 08 09:41:58.243703 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:41:58.243671 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:02.372983 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:02.372955 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:42:07.250362 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:07.250311 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:07.352335 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:07.352295 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:13.569103 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:13.569064 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:15.462299 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:15.462265 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:15.563180 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:15.563147 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:37.388243 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:37.388198 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:41.347143 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:41.347110 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:56.963522 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:56.963432 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:42:57.088869 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:42:57.088832 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:43:21.551571 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:43:21.551537 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:43:28.024482 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:43:28.024444 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:44:42.552048 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:44:42.551965 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:44:43.624557 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:44:43.624519 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:44:50.887639 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:44:50.887595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:45:00.177562 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.177531 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29725065-tqvd7"] Jul 08 09:45:00.181521 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.181498 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s"] Jul 08 09:45:00.181680 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.181618 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:00.185057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.185038 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v"] Jul 08 09:45:00.185181 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.185148 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:00.188257 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.188240 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:00.222121 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.222097 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s"] Jul 08 09:45:00.225455 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.225427 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29725065-tqvd7"] Jul 08 09:45:00.242476 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.242453 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v"] Jul 08 09:45:00.330631 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.330587 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54dw\" (UniqueName: \"kubernetes.io/projected/0e45c9d0-e492-4ed7-850c-62b3a862a9fc-kube-api-access-p54dw\") pod \"maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s\" (UID: \"0e45c9d0-e492-4ed7-850c-62b3a862a9fc\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:00.330844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.330770 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8695p\" (UniqueName: \"kubernetes.io/projected/8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b-kube-api-access-8695p\") pod \"maas-api-key-cleanup-29725065-tqvd7\" (UID: \"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b\") " pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:00.330844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.330809 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmq2s\" (UniqueName: \"kubernetes.io/projected/af3ec1a9-be75-4e74-affa-ea306e5f6183-kube-api-access-wmq2s\") pod \"maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v\" (UID: \"af3ec1a9-be75-4e74-affa-ea306e5f6183\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:00.431800 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.431699 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8695p\" (UniqueName: \"kubernetes.io/projected/8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b-kube-api-access-8695p\") pod \"maas-api-key-cleanup-29725065-tqvd7\" (UID: \"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b\") " pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:00.431800 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.431754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmq2s\" (UniqueName: \"kubernetes.io/projected/af3ec1a9-be75-4e74-affa-ea306e5f6183-kube-api-access-wmq2s\") pod \"maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v\" (UID: \"af3ec1a9-be75-4e74-affa-ea306e5f6183\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:00.432068 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.431854 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54dw\" (UniqueName: \"kubernetes.io/projected/0e45c9d0-e492-4ed7-850c-62b3a862a9fc-kube-api-access-p54dw\") pod \"maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s\" (UID: \"0e45c9d0-e492-4ed7-850c-62b3a862a9fc\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:00.441292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.441268 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8695p\" (UniqueName: \"kubernetes.io/projected/8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b-kube-api-access-8695p\") pod \"maas-api-key-cleanup-29725065-tqvd7\" (UID: \"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b\") " pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:00.443391 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.443358 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmq2s\" (UniqueName: \"kubernetes.io/projected/af3ec1a9-be75-4e74-affa-ea306e5f6183-kube-api-access-wmq2s\") pod \"maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v\" (UID: \"af3ec1a9-be75-4e74-affa-ea306e5f6183\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:00.443531 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.443509 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54dw\" (UniqueName: \"kubernetes.io/projected/0e45c9d0-e492-4ed7-850c-62b3a862a9fc-kube-api-access-p54dw\") pod \"maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s\" (UID: \"0e45c9d0-e492-4ed7-850c-62b3a862a9fc\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:00.493953 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.493897 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:00.501263 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.501245 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:00.507031 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.507011 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:00.663470 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.663423 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29725065-tqvd7"] Jul 08 09:45:00.907645 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.907608 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v"] Jul 08 09:45:00.909791 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:00.909764 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s"] Jul 08 09:45:00.927799 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:45:00.925725 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3ec1a9_be75_4e74_affa_ea306e5f6183.slice/crio-d6d9e0e2dc740d43546ef7c97a7a46c53e9b1733d3373038bcd5baf3ddbd216c WatchSource:0}: Error finding container d6d9e0e2dc740d43546ef7c97a7a46c53e9b1733d3373038bcd5baf3ddbd216c: Status 404 returned error can't find the container with id d6d9e0e2dc740d43546ef7c97a7a46c53e9b1733d3373038bcd5baf3ddbd216c Jul 08 09:45:01.389252 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.389206 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" event={"ID":"af3ec1a9-be75-4e74-affa-ea306e5f6183","Type":"ContainerStarted","Data":"0993cc39787e9cfe4ccfcaca69654d298a98c6e7cd925db0243f00d005f2435a"} Jul 08 09:45:01.389663 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.389268 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" event={"ID":"af3ec1a9-be75-4e74-affa-ea306e5f6183","Type":"ContainerStarted","Data":"d6d9e0e2dc740d43546ef7c97a7a46c53e9b1733d3373038bcd5baf3ddbd216c"} Jul 08 09:45:01.390792 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.390763 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" event={"ID":"0e45c9d0-e492-4ed7-850c-62b3a862a9fc","Type":"ContainerStarted","Data":"b09989d0f65ec45f957977a43d7390623100a871a2c26e74412fd8773dc9bc16"} Jul 08 09:45:01.390996 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.390812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" event={"ID":"0e45c9d0-e492-4ed7-850c-62b3a862a9fc","Type":"ContainerStarted","Data":"a353879dd6ae5d7d5e6f75f0c7fd96fa608f61261081c21803a6ae8a6dfa6ee0"} Jul 08 09:45:01.392218 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.392197 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" event={"ID":"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b","Type":"ContainerStarted","Data":"c3248c8f7a825c46b439728f04e2ac6228b83796bf27a28403cf54e3b7f4bdd4"} Jul 08 09:45:01.392312 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.392252 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" event={"ID":"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b","Type":"ContainerStarted","Data":"66d914c6401828090222cf6ef7e24d9a290ac5594eca043014465684f8259870"} Jul 08 09:45:01.413768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.413721 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" podStartSLOduration=1.4137066919999999 podStartE2EDuration="1.413706692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:45:01.409798238 +0000 UTC m=+2279.764809799" watchObservedRunningTime="2026-07-08 09:45:01.413706692 +0000 UTC m=+2279.768718254" Jul 08 09:45:01.430404 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.430361 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" podStartSLOduration=1.4303478969999999 podStartE2EDuration="1.430347897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:45:01.429484586 +0000 UTC m=+2279.784496147" watchObservedRunningTime="2026-07-08 09:45:01.430347897 +0000 UTC m=+2279.785359483" Jul 08 09:45:01.452008 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:01.451870 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" podStartSLOduration=1.451853586 podStartE2EDuration="1.451853586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 09:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:45:01.450692943 +0000 UTC m=+2279.805704505" watchObservedRunningTime="2026-07-08 09:45:01.451853586 +0000 UTC m=+2279.806865148" Jul 08 09:45:02.399572 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:02.399540 2572 generic.go:356] "Generic (PLEG): container finished" podID="0e45c9d0-e492-4ed7-850c-62b3a862a9fc" containerID="b09989d0f65ec45f957977a43d7390623100a871a2c26e74412fd8773dc9bc16" exitCode=0 Jul 08 09:45:02.400095 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:02.399615 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" event={"ID":"0e45c9d0-e492-4ed7-850c-62b3a862a9fc","Type":"ContainerDied","Data":"b09989d0f65ec45f957977a43d7390623100a871a2c26e74412fd8773dc9bc16"} Jul 08 09:45:02.401203 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:02.401177 2572 generic.go:356] "Generic (PLEG): container finished" podID="8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b" containerID="c3248c8f7a825c46b439728f04e2ac6228b83796bf27a28403cf54e3b7f4bdd4" exitCode=0 Jul 08 09:45:02.401338 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:02.401256 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" event={"ID":"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b","Type":"ContainerDied","Data":"c3248c8f7a825c46b439728f04e2ac6228b83796bf27a28403cf54e3b7f4bdd4"} Jul 08 09:45:02.402768 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:02.402747 2572 generic.go:356] "Generic (PLEG): container finished" podID="af3ec1a9-be75-4e74-affa-ea306e5f6183" containerID="0993cc39787e9cfe4ccfcaca69654d298a98c6e7cd925db0243f00d005f2435a" exitCode=0 Jul 08 09:45:02.402863 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:02.402784 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" event={"ID":"af3ec1a9-be75-4e74-affa-ea306e5f6183","Type":"ContainerDied","Data":"0993cc39787e9cfe4ccfcaca69654d298a98c6e7cd925db0243f00d005f2435a"} Jul 08 09:45:03.563008 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.562959 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:03.593866 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.593833 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:03.597279 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.597252 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:03.665350 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.665264 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p54dw\" (UniqueName: \"kubernetes.io/projected/0e45c9d0-e492-4ed7-850c-62b3a862a9fc-kube-api-access-p54dw\") pod \"0e45c9d0-e492-4ed7-850c-62b3a862a9fc\" (UID: \"0e45c9d0-e492-4ed7-850c-62b3a862a9fc\") " Jul 08 09:45:03.665350 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.665325 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmq2s\" (UniqueName: \"kubernetes.io/projected/af3ec1a9-be75-4e74-affa-ea306e5f6183-kube-api-access-wmq2s\") pod \"af3ec1a9-be75-4e74-affa-ea306e5f6183\" (UID: \"af3ec1a9-be75-4e74-affa-ea306e5f6183\") " Jul 08 09:45:03.665537 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.665418 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8695p\" (UniqueName: \"kubernetes.io/projected/8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b-kube-api-access-8695p\") pod \"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b\" (UID: \"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b\") " Jul 08 09:45:03.667470 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.667439 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af3ec1a9-be75-4e74-affa-ea306e5f6183-kube-api-access-wmq2s" (OuterVolumeSpecName: "kube-api-access-wmq2s") pod "af3ec1a9-be75-4e74-affa-ea306e5f6183" (UID: "af3ec1a9-be75-4e74-affa-ea306e5f6183"). InnerVolumeSpecName "kube-api-access-wmq2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:45:03.667837 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.667816 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e45c9d0-e492-4ed7-850c-62b3a862a9fc-kube-api-access-p54dw" (OuterVolumeSpecName: "kube-api-access-p54dw") pod "0e45c9d0-e492-4ed7-850c-62b3a862a9fc" (UID: "0e45c9d0-e492-4ed7-850c-62b3a862a9fc"). InnerVolumeSpecName "kube-api-access-p54dw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:45:03.667969 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.667856 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b-kube-api-access-8695p" (OuterVolumeSpecName: "kube-api-access-8695p") pod "8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b" (UID: "8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b"). InnerVolumeSpecName "kube-api-access-8695p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:45:03.766583 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.766550 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8695p\" (UniqueName: \"kubernetes.io/projected/8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b-kube-api-access-8695p\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:45:03.766583 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.766583 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p54dw\" (UniqueName: \"kubernetes.io/projected/0e45c9d0-e492-4ed7-850c-62b3a862a9fc-kube-api-access-p54dw\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:45:03.766583 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:03.766595 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmq2s\" (UniqueName: \"kubernetes.io/projected/af3ec1a9-be75-4e74-affa-ea306e5f6183-kube-api-access-wmq2s\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:45:04.414857 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.414830 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" Jul 08 09:45:04.415049 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.414827 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s" event={"ID":"0e45c9d0-e492-4ed7-850c-62b3a862a9fc","Type":"ContainerDied","Data":"a353879dd6ae5d7d5e6f75f0c7fd96fa608f61261081c21803a6ae8a6dfa6ee0"} Jul 08 09:45:04.415049 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.414995 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a353879dd6ae5d7d5e6f75f0c7fd96fa608f61261081c21803a6ae8a6dfa6ee0" Jul 08 09:45:04.416244 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.416220 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" Jul 08 09:45:04.416368 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.416239 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29725065-tqvd7" event={"ID":"8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b","Type":"ContainerDied","Data":"66d914c6401828090222cf6ef7e24d9a290ac5594eca043014465684f8259870"} Jul 08 09:45:04.416368 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.416280 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66d914c6401828090222cf6ef7e24d9a290ac5594eca043014465684f8259870" Jul 08 09:45:04.417574 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.417542 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" event={"ID":"af3ec1a9-be75-4e74-affa-ea306e5f6183","Type":"ContainerDied","Data":"d6d9e0e2dc740d43546ef7c97a7a46c53e9b1733d3373038bcd5baf3ddbd216c"} Jul 08 09:45:04.417677 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.417577 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6d9e0e2dc740d43546ef7c97a7a46c53e9b1733d3373038bcd5baf3ddbd216c" Jul 08 09:45:04.417677 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:04.417578 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v" Jul 08 09:45:08.432547 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:08.432514 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:45:16.493332 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:45:16.493293 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:46:01.354534 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:46:01.354501 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:46:03.233145 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:46:03.233107 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:46:11.437646 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:46:11.437610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:46:22.423923 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:46:22.423854 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:46:22.525573 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:46:22.525523 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:00.337411 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:00.337371 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:07.985826 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:07.985787 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:08.086199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:08.086141 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:09.488020 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:09.487983 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:16.328077 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:16.328043 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:16.427675 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:16.427619 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:47.726189 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:47.726091 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:48.540749 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:48.540716 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:49.680929 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:49.680867 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:50.071011 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.070888 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:50.184449 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.184419 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:50.285107 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.285065 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:50.437395 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.437361 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw"] Jul 08 09:47:50.437598 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.437563 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" podUID="98b4fb87-3ee3-4c7c-b306-f65e5c204a20" containerName="maas-api" containerID="cri-o://bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392" gracePeriod=30 Jul 08 09:47:50.472184 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.472148 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:47:50.488144 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.488120 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s"] Jul 08 09:47:50.492171 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.492147 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-8c4427-29725065-8bw8s"] Jul 08 09:47:50.687750 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.687694 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:47:50.779444 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.779410 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-maas-api-tls\") pod \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " Jul 08 09:47:50.779625 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.779465 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28772\" (UniqueName: \"kubernetes.io/projected/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-kube-api-access-28772\") pod \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " Jul 08 09:47:50.779625 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.779539 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-openshift-service-ca\") pod \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\" (UID: \"98b4fb87-3ee3-4c7c-b306-f65e5c204a20\") " Jul 08 09:47:50.779964 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.779938 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98b4fb87-3ee3-4c7c-b306-f65e5c204a20" (UID: "98b4fb87-3ee3-4c7c-b306-f65e5c204a20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:47:50.781554 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.781531 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98b4fb87-3ee3-4c7c-b306-f65e5c204a20" (UID: "98b4fb87-3ee3-4c7c-b306-f65e5c204a20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:47:50.781625 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.781580 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-kube-api-access-28772" (OuterVolumeSpecName: "kube-api-access-28772") pod "98b4fb87-3ee3-4c7c-b306-f65e5c204a20" (UID: "98b4fb87-3ee3-4c7c-b306-f65e5c204a20"). InnerVolumeSpecName "kube-api-access-28772". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:47:50.880967 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.880934 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-maas-api-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:47:50.880967 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.880962 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28772\" (UniqueName: \"kubernetes.io/projected/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-kube-api-access-28772\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:47:50.881134 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:50.880979 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b4fb87-3ee3-4c7c-b306-f65e5c204a20-openshift-service-ca\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:47:51.368808 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.368775 2572 generic.go:356] "Generic (PLEG): container finished" podID="98b4fb87-3ee3-4c7c-b306-f65e5c204a20" containerID="bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392" exitCode=0 Jul 08 09:47:51.369006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.368822 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" Jul 08 09:47:51.369006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.368867 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" event={"ID":"98b4fb87-3ee3-4c7c-b306-f65e5c204a20","Type":"ContainerDied","Data":"bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392"} Jul 08 09:47:51.369006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.368948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw" event={"ID":"98b4fb87-3ee3-4c7c-b306-f65e5c204a20","Type":"ContainerDied","Data":"8f65b5afd1e463e6d9ba9b3a4249c1b590f14d62750b7a33add92461280a12a3"} Jul 08 09:47:51.369006 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.368977 2572 scope.go:117] "RemoveContainer" containerID="bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392" Jul 08 09:47:51.378487 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.378465 2572 scope.go:117] "RemoveContainer" containerID="bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392" Jul 08 09:47:51.378714 ip-10-0-140-252 kubenswrapper[2572]: E0708 09:47:51.378694 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392\": container with ID starting with bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392 not found: ID does not exist" containerID="bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392" Jul 08 09:47:51.378761 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.378722 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392"} err="failed to get container status \"bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392\": rpc error: code = NotFound desc = could not find container \"bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392\": container with ID starting with bd015f2a67f4afea687db24a7f3f75f49661f11ed6d606988a6e01529e5f9392 not found: ID does not exist" Jul 08 09:47:51.393606 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.393584 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw"] Jul 08 09:47:51.399735 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:51.399714 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c4427-6cb8fd5f87-ztzpw"] Jul 08 09:47:52.176332 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:52.176301 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e45c9d0-e492-4ed7-850c-62b3a862a9fc" path="/var/lib/kubelet/pods/0e45c9d0-e492-4ed7-850c-62b3a862a9fc/volumes" Jul 08 09:47:52.176930 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:52.176889 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b4fb87-3ee3-4c7c-b306-f65e5c204a20" path="/var/lib/kubelet/pods/98b4fb87-3ee3-4c7c-b306-f65e5c204a20/volumes" Jul 08 09:47:55.240165 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:47:55.240127 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:09.691070 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:09.691030 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:09.794130 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:09.794093 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:32.778096 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:32.778063 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:33.087423 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:33.087341 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:33.319268 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:33.319237 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:33.606599 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:33.606558 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:34.014767 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.014673 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:34.135306 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.135268 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:34.235154 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.235120 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:34.428183 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.428145 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t"] Jul 08 09:48:34.428445 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.428413 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" podUID="fa7b0996-9f9d-4319-b0f9-f3fae96605cb" containerName="maas-api" containerID="cri-o://c9943f050489f802d48d0f413a61c766febc1cb34254f87ea0cd6fbe067d81c0" gracePeriod=30 Jul 08 09:48:34.465116 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.465079 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:34.478510 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.478485 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v"] Jul 08 09:48:34.487823 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.487796 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-f0ff0b-29725065-txc2v"] Jul 08 09:48:34.618959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.618891 2572 generic.go:356] "Generic (PLEG): container finished" podID="fa7b0996-9f9d-4319-b0f9-f3fae96605cb" containerID="c9943f050489f802d48d0f413a61c766febc1cb34254f87ea0cd6fbe067d81c0" exitCode=0 Jul 08 09:48:34.618959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.618939 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" event={"ID":"fa7b0996-9f9d-4319-b0f9-f3fae96605cb","Type":"ContainerDied","Data":"c9943f050489f802d48d0f413a61c766febc1cb34254f87ea0cd6fbe067d81c0"} Jul 08 09:48:34.685772 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.685707 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:48:34.807313 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.807279 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls\") pod \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " Jul 08 09:48:34.807488 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.807422 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-openshift-service-ca\") pod \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " Jul 08 09:48:34.807488 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.807465 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgrbm\" (UniqueName: \"kubernetes.io/projected/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-kube-api-access-xgrbm\") pod \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\" (UID: \"fa7b0996-9f9d-4319-b0f9-f3fae96605cb\") " Jul 08 09:48:34.807848 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.807819 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa7b0996-9f9d-4319-b0f9-f3fae96605cb" (UID: "fa7b0996-9f9d-4319-b0f9-f3fae96605cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 09:48:34.809351 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.809324 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa7b0996-9f9d-4319-b0f9-f3fae96605cb" (UID: "fa7b0996-9f9d-4319-b0f9-f3fae96605cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 09:48:34.809453 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.809436 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-kube-api-access-xgrbm" (OuterVolumeSpecName: "kube-api-access-xgrbm") pod "fa7b0996-9f9d-4319-b0f9-f3fae96605cb" (UID: "fa7b0996-9f9d-4319-b0f9-f3fae96605cb"). InnerVolumeSpecName "kube-api-access-xgrbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 09:48:34.909246 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.909207 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-openshift-service-ca\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:48:34.909246 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.909238 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgrbm\" (UniqueName: \"kubernetes.io/projected/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-kube-api-access-xgrbm\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:48:34.909246 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:34.909248 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa7b0996-9f9d-4319-b0f9-f3fae96605cb-maas-api-tls\") on node \"ip-10-0-140-252.ec2.internal\" DevicePath \"\"" Jul 08 09:48:35.626155 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:35.626127 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" Jul 08 09:48:35.626552 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:35.626158 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t" event={"ID":"fa7b0996-9f9d-4319-b0f9-f3fae96605cb","Type":"ContainerDied","Data":"2a3ae82f518a96a99c46b6fc319e5768357f756ba2557f8cb1d88792d63df1a8"} Jul 08 09:48:35.626552 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:35.626233 2572 scope.go:117] "RemoveContainer" containerID="c9943f050489f802d48d0f413a61c766febc1cb34254f87ea0cd6fbe067d81c0" Jul 08 09:48:35.652933 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:35.652883 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t"] Jul 08 09:48:35.657291 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:35.657268 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f0ff0b-5d778b6c88-n572t"] Jul 08 09:48:36.177124 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:36.177094 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af3ec1a9-be75-4e74-affa-ea306e5f6183" path="/var/lib/kubelet/pods/af3ec1a9-be75-4e74-affa-ea306e5f6183/volumes" Jul 08 09:48:36.177481 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:36.177468 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa7b0996-9f9d-4319-b0f9-f3fae96605cb" path="/var/lib/kubelet/pods/fa7b0996-9f9d-4319-b0f9-f3fae96605cb/volumes" Jul 08 09:48:40.623452 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:40.623412 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:40.697741 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:40.697701 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:40.902520 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:40.902430 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:41.137837 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:41.137784 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:46.494615 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:46.494567 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-v6qbn_28d7ead6-d196-460d-9fe8-800d533065ac/manager/0.log" Jul 08 09:48:46.622687 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:46.622658 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bc2zp_3672feac-433d-4794-b5b2-8e19fb40be55/manager/0.log" Jul 08 09:48:46.881104 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:46.881069 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29725050-g2jsn_4a67b618-74cb-43eb-ac6e-12f46709425c/cleanup/0.log" Jul 08 09:48:47.010437 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:47.010409 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29725065-tqvd7_8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b/cleanup/0.log" Jul 08 09:48:47.160450 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:47.160320 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d6cbf4fcb-gmp4d_48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63/manager/0.log" Jul 08 09:48:47.279106 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:47.279074 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cc9fx_19000ff5-718e-4a83-a56d-e01d22236c84/server/0.log" Jul 08 09:48:47.414563 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:47.414486 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-x9c7p_8503c860-2306-4b4c-9b33-f0c86e69d98f/manager/0.log" Jul 08 09:48:47.700532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:47.700457 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-qg4hp_c08bd53d-eb89-4df8-9e20-2961dc1f3034/copy-manifests/0.log" Jul 08 09:48:47.710565 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:47.710536 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-qg4hp_c08bd53d-eb89-4df8-9e20-2961dc1f3034/manager/0.log" Jul 08 09:48:49.516879 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:49.516817 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zbqfm_f3d6f712-e668-4989-8f08-d1ad02d937ca/manager/0.log" Jul 08 09:48:49.644429 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:49.644399 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qnrvw_0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a/manager/0.log" Jul 08 09:48:49.763659 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:49.763627 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fwnpr_bde984ea-c427-45ba-b5a7-8771de4ab293/kuadrant-console-plugin/0.log" Jul 08 09:48:49.895844 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:49.895792 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-cl526_32e2ca96-83dc-4272-8743-fb7531f0ef09/registry-server/0.log" Jul 08 09:48:50.027841 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:50.027813 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rrtqv_13e07f80-e221-4b96-a6ff-03d15f3700d2/manager/0.log" Jul 08 09:48:50.297180 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:50.297094 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-47q2w_ec78e5c0-6923-4824-bfd1-60c1b2d854d5/manager/0.log" Jul 08 09:48:50.670874 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:50.670844 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nw4h5_e4514cb1-0237-4496-a79f-a194e578c664/istio-proxy/0.log" Jul 08 09:48:50.926504 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:50.926417 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5458fdb769-sv77r_d12c84a8-3768-4dbe-8fa3-1b58d3c9f823/kube-auth-proxy/0.log" Jul 08 09:48:51.182273 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:51.182198 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/1.log" Jul 08 09:48:51.440640 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:51.440562 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hj89_7ce96574-14b8-488d-86e2-a1ed910f1a31/payload-processing/0.log" Jul 08 09:48:51.942854 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:51.942817 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4tpnc_f3251e3d-e7be-4126-adca-d1c0ca0d0cab/main/0.log" Jul 08 09:48:52.207857 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:52.207773 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2tm9g_794a4e41-fbe5-4857-8f6c-aa48e4d64f66/main/0.log" Jul 08 09:48:53.254129 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:53.254031 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:57.459792 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:57.459748 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:48:57.549492 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:48:57.549450 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" limit=1024 Jul 08 09:49:00.740896 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:00.740866 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sl4vp_862d5d42-03dd-41be-a8c4-923c31d4ba80/global-pull-secret-syncer/0.log" Jul 08 09:49:00.810236 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:00.810201 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8299v_904f4b1b-a4c1-4bcf-bb43-fa4a99ed94df/konnectivity-agent/0.log" Jul 08 09:49:00.989959 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:00.989928 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-252.ec2.internal_5a24e32bf99f9fe88f14a0d838a9684d/haproxy/0.log" Jul 08 09:49:06.210994 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:06.210958 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zbqfm_f3d6f712-e668-4989-8f08-d1ad02d937ca/manager/0.log" Jul 08 09:49:06.273878 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:06.273848 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qnrvw_0148feb3-6cbb-4a98-b5ff-4d063a3e1e4a/manager/0.log" Jul 08 09:49:06.302429 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:06.302404 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fwnpr_bde984ea-c427-45ba-b5a7-8771de4ab293/kuadrant-console-plugin/0.log" Jul 08 09:49:06.359259 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:06.359230 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-cl526_32e2ca96-83dc-4272-8743-fb7531f0ef09/registry-server/0.log" Jul 08 09:49:06.489150 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:06.489125 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rrtqv_13e07f80-e221-4b96-a6ff-03d15f3700d2/manager/0.log" Jul 08 09:49:06.642821 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:06.642778 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-47q2w_ec78e5c0-6923-4824-bfd1-60c1b2d854d5/manager/0.log" Jul 08 09:49:08.384120 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.384092 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/alertmanager/0.log" Jul 08 09:49:08.408864 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.408830 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/config-reloader/0.log" Jul 08 09:49:08.434410 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.434381 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/kube-rbac-proxy-web/0.log" Jul 08 09:49:08.462334 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.462266 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/kube-rbac-proxy/0.log" Jul 08 09:49:08.487280 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.487246 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/kube-rbac-proxy-metric/0.log" Jul 08 09:49:08.512613 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.512584 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/prom-label-proxy/0.log" Jul 08 09:49:08.535715 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.535689 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a4c6d5d-8922-4c81-81f0-fc38dad66bad/init-config-reloader/0.log" Jul 08 09:49:08.623811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.623779 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8qh2v_62cf7623-90e6-4c31-8c79-5d5349af1c1f/kube-state-metrics/0.log" Jul 08 09:49:08.651493 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.651465 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8qh2v_62cf7623-90e6-4c31-8c79-5d5349af1c1f/kube-rbac-proxy-main/0.log" Jul 08 09:49:08.681200 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.681176 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-8qh2v_62cf7623-90e6-4c31-8c79-5d5349af1c1f/kube-rbac-proxy-self/0.log" Jul 08 09:49:08.716457 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.716377 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8699564ddd-krr97_efd395c9-49c7-4b95-a7e6-c8d40d5cb19c/metrics-server/0.log" Jul 08 09:49:08.960255 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.960229 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k2q9v_8787e42a-b40c-46e5-9a30-b13010103b67/node-exporter/0.log" Jul 08 09:49:08.984845 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:08.984772 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k2q9v_8787e42a-b40c-46e5-9a30-b13010103b67/kube-rbac-proxy/0.log" Jul 08 09:49:09.011087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.011065 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k2q9v_8787e42a-b40c-46e5-9a30-b13010103b67/init-textfile/0.log" Jul 08 09:49:09.041575 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.041555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nn5d7_b0de23a3-a3fc-4d89-80f4-415886dd6e02/kube-rbac-proxy-main/0.log" Jul 08 09:49:09.066730 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.066705 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nn5d7_b0de23a3-a3fc-4d89-80f4-415886dd6e02/kube-rbac-proxy-self/0.log" Jul 08 09:49:09.093855 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.093819 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nn5d7_b0de23a3-a3fc-4d89-80f4-415886dd6e02/openshift-state-metrics/0.log" Jul 08 09:49:09.131412 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.131377 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/prometheus/0.log" Jul 08 09:49:09.155486 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.155456 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/config-reloader/0.log" Jul 08 09:49:09.187055 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.187028 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/thanos-sidecar/0.log" Jul 08 09:49:09.211311 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.211286 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/kube-rbac-proxy-web/0.log" Jul 08 09:49:09.237435 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.237358 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/kube-rbac-proxy/0.log" Jul 08 09:49:09.263533 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.263505 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/kube-rbac-proxy-thanos/0.log" Jul 08 09:49:09.291827 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.291803 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1718182c-492c-48f4-ae5c-38c3ce460403/init-config-reloader/0.log" Jul 08 09:49:09.328577 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.328550 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-9s2wt_0f69d47e-67ab-4cd8-b2e2-d9ce835c8910/prometheus-operator/0.log" Jul 08 09:49:09.352824 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:09.352802 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-9s2wt_0f69d47e-67ab-4cd8-b2e2-d9ce835c8910/kube-rbac-proxy/0.log" Jul 08 09:49:10.783086 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:10.783060 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xhdkq_d7c2a543-b027-45f4-b482-15db05f448a3/networking-console-plugin/0.log" Jul 08 09:49:11.297402 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297363 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt"] Jul 08 09:49:11.297820 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297807 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa7b0996-9f9d-4319-b0f9-f3fae96605cb" containerName="maas-api" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297822 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa7b0996-9f9d-4319-b0f9-f3fae96605cb" containerName="maas-api" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297836 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b" containerName="cleanup" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297841 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b" containerName="cleanup" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297860 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e45c9d0-e492-4ed7-850c-62b3a862a9fc" containerName="cleanup" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297867 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e45c9d0-e492-4ed7-850c-62b3a862a9fc" containerName="cleanup" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297877 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3ec1a9-be75-4e74-affa-ea306e5f6183" containerName="cleanup" Jul 08 09:49:11.297881 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297882 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3ec1a9-be75-4e74-affa-ea306e5f6183" containerName="cleanup" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297892 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b4fb87-3ee3-4c7c-b306-f65e5c204a20" containerName="maas-api" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297897 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b4fb87-3ee3-4c7c-b306-f65e5c204a20" containerName="maas-api" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297973 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa7b0996-9f9d-4319-b0f9-f3fae96605cb" containerName="maas-api" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297982 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3ec1a9-be75-4e74-affa-ea306e5f6183" containerName="cleanup" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297988 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e45c9d0-e492-4ed7-850c-62b3a862a9fc" containerName="cleanup" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.297995 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b4fb87-3ee3-4c7c-b306-f65e5c204a20" containerName="maas-api" Jul 08 09:49:11.298133 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.298004 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b" containerName="cleanup" Jul 08 09:49:11.301125 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.301106 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.303910 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.303876 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prsl4\"/\"default-dockercfg-6smtf\"" Jul 08 09:49:11.304925 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.304889 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prsl4\"/\"kube-root-ca.crt\"" Jul 08 09:49:11.305021 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.304930 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prsl4\"/\"openshift-service-ca.crt\"" Jul 08 09:49:11.311026 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.311003 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt"] Jul 08 09:49:11.479571 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.479539 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-sys\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.479571 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.479584 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-proc\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.479789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.479605 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-lib-modules\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.479789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.479697 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zwb2\" (UniqueName: \"kubernetes.io/projected/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-kube-api-access-4zwb2\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.479789 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.479736 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-podres\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.580841 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580814 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-sys\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.580956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-proc\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.580956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580872 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-lib-modules\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.580956 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580940 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zwb2\" (UniqueName: \"kubernetes.io/projected/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-kube-api-access-4zwb2\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.581087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580955 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-proc\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.581087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580956 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-sys\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.581087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.580985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-podres\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.581087 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.581058 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-lib-modules\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.581221 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.581136 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-podres\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.593987 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.593966 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zwb2\" (UniqueName: \"kubernetes.io/projected/5929f7dd-fcdd-4d7c-a0b0-69558d5401be-kube-api-access-4zwb2\") pod \"perf-node-gather-daemonset-x72rt\" (UID: \"5929f7dd-fcdd-4d7c-a0b0-69558d5401be\") " pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.612950 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.612924 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:11.749200 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.749174 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt"] Jul 08 09:49:11.750790 ip-10-0-140-252 kubenswrapper[2572]: W0708 09:49:11.750762 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5929f7dd_fcdd_4d7c_a0b0_69558d5401be.slice/crio-e68fb85f06749ef52d6b4857d40a523c887b77feea08ec5cc30fd7606370623e WatchSource:0}: Error finding container e68fb85f06749ef52d6b4857d40a523c887b77feea08ec5cc30fd7606370623e: Status 404 returned error can't find the container with id e68fb85f06749ef52d6b4857d40a523c887b77feea08ec5cc30fd7606370623e Jul 08 09:49:11.826384 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:11.826351 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" event={"ID":"5929f7dd-fcdd-4d7c-a0b0-69558d5401be","Type":"ContainerStarted","Data":"e68fb85f06749ef52d6b4857d40a523c887b77feea08ec5cc30fd7606370623e"} Jul 08 09:49:12.696472 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:12.696442 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-5svmn_42d8150b-fb0b-4a6e-bae3-0db02436b2a5/volume-data-source-validator/0.log" Jul 08 09:49:12.832350 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:12.832309 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" event={"ID":"5929f7dd-fcdd-4d7c-a0b0-69558d5401be","Type":"ContainerStarted","Data":"284b11d31f74dc2dda260980c49e45a998f4db37aa1637f592ae3f623e3844be"} Jul 08 09:49:12.832730 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:12.832420 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:12.850994 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:12.850949 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" podStartSLOduration=1.850934277 podStartE2EDuration="1.850934277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 09:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 09:49:12.849474829 +0000 UTC m=+2531.204486393" watchObservedRunningTime="2026-07-08 09:49:12.850934277 +0000 UTC m=+2531.205945839" Jul 08 09:49:13.887887 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:13.887855 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zf8b5_fa74e311-430b-42f8-bc39-dd43f477309c/dns/0.log" Jul 08 09:49:13.914526 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:13.914496 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zf8b5_fa74e311-430b-42f8-bc39-dd43f477309c/kube-rbac-proxy/0.log" Jul 08 09:49:14.002199 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:14.002171 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-486wp_e77a0967-6023-4cc7-98da-290dcea1391a/dns-node-resolver/0.log" Jul 08 09:49:14.664841 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:14.664808 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f45cdf87-khsjs_9843a1ff-3861-4e32-ba99-9881307906f2/registry/0.log" Jul 08 09:49:14.766620 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:14.766593 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-crgh2_ad916d36-3a34-4b04-96dd-d831d52ea3c6/node-ca/0.log" Jul 08 09:49:15.782541 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:15.782504 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nw4h5_e4514cb1-0237-4496-a79f-a194e578c664/istio-proxy/0.log" Jul 08 09:49:15.914134 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:15.914107 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5458fdb769-sv77r_d12c84a8-3768-4dbe-8fa3-1b58d3c9f823/kube-auth-proxy/0.log" Jul 08 09:49:15.978264 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:15.978232 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/0.log" Jul 08 09:49:16.031208 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:16.031182 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/1.log" Jul 08 09:49:16.142588 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:16.142554 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hj89_7ce96574-14b8-488d-86e2-a1ed910f1a31/payload-processing/0.log" Jul 08 09:49:16.949811 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:16.949784 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jfrkc_b8c50830-ba2b-427f-ad54-786c4d84b3f7/serve-healthcheck-canary/0.log" Jul 08 09:49:17.648027 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:17.647995 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k78ww_48daeaa4-f575-471c-8ba1-623ba4be13ff/kube-rbac-proxy/0.log" Jul 08 09:49:17.673400 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:17.673373 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k78ww_48daeaa4-f575-471c-8ba1-623ba4be13ff/exporter/0.log" Jul 08 09:49:17.700475 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:17.700446 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k78ww_48daeaa4-f575-471c-8ba1-623ba4be13ff/extractor/0.log" Jul 08 09:49:18.847689 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:18.847662 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prsl4/perf-node-gather-daemonset-x72rt" Jul 08 09:49:20.114012 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.113971 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-v6qbn_28d7ead6-d196-460d-9fe8-800d533065ac/manager/0.log" Jul 08 09:49:20.337456 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.337423 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bc2zp_3672feac-433d-4794-b5b2-8e19fb40be55/manager/0.log" Jul 08 09:49:20.462059 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.461965 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29725050-g2jsn_4a67b618-74cb-43eb-ac6e-12f46709425c/cleanup/0.log" Jul 08 09:49:20.503000 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.502976 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29725065-tqvd7_8a2f0bd1-8503-4df3-8b9d-40f7529a5c9b/cleanup/0.log" Jul 08 09:49:20.669435 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.669398 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d6cbf4fcb-gmp4d_48d8f8e9-dc1e-4d4f-ac9e-2907ff04cf63/manager/0.log" Jul 08 09:49:20.698967 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.698939 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cc9fx_19000ff5-718e-4a83-a56d-e01d22236c84/server/0.log" Jul 08 09:49:20.828538 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.828440 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-x9c7p_8503c860-2306-4b4c-9b33-f0c86e69d98f/manager/0.log" Jul 08 09:49:20.959705 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.959674 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-qg4hp_c08bd53d-eb89-4df8-9e20-2961dc1f3034/manager/0.log" Jul 08 09:49:20.988202 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:20.988174 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d6485f98-qg4hp_c08bd53d-eb89-4df8-9e20-2961dc1f3034/copy-manifests/0.log" Jul 08 09:49:21.759078 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:21.759053 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nw4h5_e4514cb1-0237-4496-a79f-a194e578c664/istio-proxy/0.log" Jul 08 09:49:21.881241 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:21.881206 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5458fdb769-sv77r_d12c84a8-3768-4dbe-8fa3-1b58d3c9f823/kube-auth-proxy/0.log" Jul 08 09:49:21.935532 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:21.935497 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/0.log" Jul 08 09:49:21.994292 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:21.994210 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/1.log" Jul 08 09:49:22.086595 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:22.086568 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hj89_7ce96574-14b8-488d-86e2-a1ed910f1a31/payload-processing/0.log" Jul 08 09:49:22.769597 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:22.769560 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tkd56_98803f93-c6cb-4f92-b0ef-e9de44fcbdfa/openshift-lws-operator/0.log" Jul 08 09:49:23.431950 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:23.431922 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4nw4h5_e4514cb1-0237-4496-a79f-a194e578c664/istio-proxy/0.log" Jul 08 09:49:23.543339 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:23.543312 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5458fdb769-sv77r_d12c84a8-3768-4dbe-8fa3-1b58d3c9f823/kube-auth-proxy/0.log" Jul 08 09:49:23.586852 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:23.586807 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/0.log" Jul 08 09:49:23.640474 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:23.640444 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xdj4x_5b8e350c-c0f2-4339-84b0-81dcb316e033/istio-proxy/1.log" Jul 08 09:49:23.734831 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:23.734736 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6hj89_7ce96574-14b8-488d-86e2-a1ed910f1a31/payload-processing/0.log" Jul 08 09:49:28.067800 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:28.067770 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-8kbql_735b79b5-2139-4483-87b3-368a07110f1e/migrator/0.log" Jul 08 09:49:28.102605 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:28.102579 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-8kbql_735b79b5-2139-4483-87b3-368a07110f1e/graceful-termination/0.log" Jul 08 09:49:28.602996 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:28.602966 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-qpjjd_048aa6fd-5839-43dd-a471-1f58a7ab8fd4/kube-storage-version-migrator-operator/1.log" Jul 08 09:49:28.604784 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:28.604761 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-qpjjd_048aa6fd-5839-43dd-a471-1f58a7ab8fd4/kube-storage-version-migrator-operator/0.log" Jul 08 09:49:29.683300 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:29.683271 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8whtj_2b2b7a91-be48-482c-8469-e35e0c9b4549/kube-multus/0.log" Jul 08 09:49:30.118057 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:30.118026 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtxtw_069db77f-888a-44c4-99b4-08d9bba7cddc/kube-multus-additional-cni-plugins/0.log" Jul 08 09:49:30.146042 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:30.146017 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtxtw_069db77f-888a-44c4-99b4-08d9bba7cddc/egress-router-binary-copy/0.log" Jul 08 09:49:30.178506 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:30.178456 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtxtw_069db77f-888a-44c4-99b4-08d9bba7cddc/cni-plugins/0.log" Jul 08 09:49:30.205730 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:30.205702 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtxtw_069db77f-888a-44c4-99b4-08d9bba7cddc/bond-cni-plugin/0.log" Jul 08 09:49:30.232118 ip-10-0-140-252 kubenswrapper[2572]: I0708 09:49:30.232088 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtxtw_069db77f-888a-44c4-99b4-08d9bba7cddc/routeoverride-cni/0.log"