Jul 20 16:43:15.339807 ip-10-0-131-206 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 20 16:43:15.339818 ip-10-0-131-206 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 20 16:43:15.339825 ip-10-0-131-206 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 20 16:43:15.340044 ip-10-0-131-206 systemd[1]: Failed to start Kubernetes Kubelet. Jul 20 16:43:25.539273 ip-10-0-131-206 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 20 16:43:25.539291 ip-10-0-131-206 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 65ae73a29dc34975a057c38711e54341 -- Jul 20 16:45:10.877644 ip-10-0-131-206 systemd[1]: Starting Kubernetes Kubelet... Jul 20 16:45:11.241033 ip-10-0-131-206 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:45:11.241033 ip-10-0-131-206 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 16:45:11.241033 ip-10-0-131-206 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:45:11.241033 ip-10-0-131-206 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 16:45:11.241033 ip-10-0-131-206 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 16:45:11.242674 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.242582 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 16:45:11.247372 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247348 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:11.247372 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247368 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:11.247372 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247372 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:11.247372 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247375 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:11.247372 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247378 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:11.247372 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247381 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247386 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247388 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247391 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247394 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247399 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247403 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247406 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247409 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247411 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247414 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247417 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247420 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247422 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247425 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247427 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247430 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247432 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247435 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:11.247601 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247438 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247440 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247443 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247445 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247448 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247450 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247453 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247459 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247461 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247464 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247466 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247468 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247471 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247474 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247476 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247480 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247483 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247486 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247488 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:11.248069 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247491 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247493 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247496 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247498 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247501 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247503 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247506 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247509 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247511 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247513 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247516 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247519 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247521 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247523 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247526 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247528 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247531 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247533 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247535 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247538 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:11.248538 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247541 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247543 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247546 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247548 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247552 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247554 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247557 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247559 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247564 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247567 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247570 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247573 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247576 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247578 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247580 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247633 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247639 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247642 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247645 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:11.249027 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247648 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247651 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247654 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247657 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247660 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247662 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247665 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247668 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247670 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247673 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247675 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247678 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247680 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247683 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247685 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247688 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.247691 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248413 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248421 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248424 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:11.249503 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248427 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248430 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248432 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248435 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248437 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248449 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248452 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248455 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248457 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248460 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248462 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248465 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248468 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248470 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248473 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248476 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248478 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248481 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248483 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:11.249995 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248486 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248488 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248490 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248493 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248495 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248498 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248500 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248503 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248505 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248508 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248510 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248513 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248515 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248518 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248521 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248524 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248526 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248529 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248531 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248534 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:11.250520 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248536 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248539 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248542 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248544 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248546 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248549 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248552 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248554 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248560 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248563 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248566 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248569 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248572 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248574 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248577 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248580 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248582 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248586 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248590 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:11.251008 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248592 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248595 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248598 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248601 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248604 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248607 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248611 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248616 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248620 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248624 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248626 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248629 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248632 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248634 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248637 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248639 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248642 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248644 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248647 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248649 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:11.251484 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248651 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248654 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248657 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248659 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248662 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248665 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248667 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248670 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248672 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248675 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248679 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248682 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248684 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248686 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248689 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248692 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248694 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.248696 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249067 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249073 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:11.251974 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249075 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249078 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249080 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249083 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249085 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249088 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249090 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249093 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249096 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249099 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249101 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249105 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249107 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249109 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249112 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249114 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249117 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249119 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249122 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:11.252490 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249125 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249128 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249130 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249132 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249135 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249137 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249140 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249142 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249144 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249147 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249149 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249152 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249155 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249157 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249160 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249162 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249165 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249167 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249170 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249172 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:11.252948 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249175 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249178 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249180 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249183 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249185 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249187 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249190 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249192 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249195 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249198 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249200 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249202 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249205 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249207 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249210 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249212 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249215 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249217 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249219 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249222 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:11.253448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249224 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249228 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249231 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249237 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249241 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249244 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249247 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249250 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249253 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249255 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249258 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249261 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249263 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249266 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249269 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249272 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249274 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249277 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249279 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:11.253932 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249282 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249284 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249287 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249289 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249292 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249294 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249297 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249299 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249302 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249318 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249321 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249323 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249326 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249328 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249330 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249333 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249336 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249339 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.249342 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249417 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249424 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 16:45:11.254414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249443 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249447 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249452 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249455 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249459 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249464 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249467 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249470 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249473 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249476 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249479 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249482 2579 flags.go:64] FLAG: --cgroup-root="" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249485 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249487 2579 flags.go:64] FLAG: --client-ca-file="" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249491 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249494 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249497 2579 flags.go:64] FLAG: --cluster-domain="" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249500 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249503 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249506 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249509 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249513 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249516 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 16:45:11.254919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249519 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249522 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249525 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249528 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249531 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249536 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249539 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249542 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249545 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249548 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249551 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249554 2579 flags.go:64] FLAG: --enable-server="true" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249557 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249561 2579 flags.go:64] FLAG: --event-burst="100" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249564 2579 flags.go:64] FLAG: --event-qps="50" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249567 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249569 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249572 2579 flags.go:64] FLAG: --eviction-hard="" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249576 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249578 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249581 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249584 2579 flags.go:64] FLAG: --eviction-soft="" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249587 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249589 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249593 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 16:45:11.255518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249596 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249598 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249601 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249604 2579 flags.go:64] FLAG: --feature-gates="" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249608 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249611 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249614 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249617 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249620 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249623 2579 flags.go:64] FLAG: --help="false" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249626 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249629 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249633 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249636 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249639 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249642 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249645 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249648 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249651 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249653 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249656 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249659 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249662 2579 flags.go:64] FLAG: --kube-reserved="" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249665 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 16:45:11.256131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249667 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249670 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249673 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249676 2579 flags.go:64] FLAG: --lock-file="" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249679 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249681 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249684 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249689 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249692 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249695 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249698 2579 flags.go:64] FLAG: --logging-format="text" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249701 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249704 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249707 2579 flags.go:64] FLAG: --manifest-url="" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249710 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249714 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249717 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249721 2579 flags.go:64] FLAG: --max-pods="110" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249724 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249727 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249731 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249733 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249736 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249739 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249742 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 16:45:11.256723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249749 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249752 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249755 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249758 2579 flags.go:64] FLAG: --pod-cidr="" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249760 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249766 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249769 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249772 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249775 2579 flags.go:64] FLAG: --port="10250" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249778 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249780 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eab9adeff269c605" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249783 2579 flags.go:64] FLAG: --qos-reserved="" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249786 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249789 2579 flags.go:64] FLAG: --register-node="true" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249792 2579 flags.go:64] FLAG: --register-with-taints="" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249795 2579 flags.go:64] FLAG: --registry-burst="10" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249798 2579 flags.go:64] FLAG: --registry-qps="5" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249801 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249804 2579 flags.go:64] FLAG: --reserved-memory="" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249807 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249810 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249813 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249815 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249818 2579 flags.go:64] FLAG: --runonce="false" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249821 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 16:45:11.257354 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249823 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249826 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249828 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249832 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249835 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249838 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249841 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249844 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249846 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249849 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249851 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249854 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249857 2579 flags.go:64] FLAG: --system-cgroups="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249859 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249864 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249867 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249870 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249873 2579 flags.go:64] FLAG: --tls-min-version="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249875 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249878 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249881 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249883 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249886 2579 flags.go:64] FLAG: --v="2" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249890 2579 flags.go:64] FLAG: --version="false" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249896 2579 flags.go:64] FLAG: --vmodule="" Jul 20 16:45:11.257958 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249900 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.249903 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250011 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250014 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250017 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250021 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250024 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250026 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250029 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250031 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250035 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250037 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250040 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250043 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250045 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250048 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250050 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250053 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250055 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250058 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:11.258625 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250060 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250063 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250065 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250068 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250070 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250073 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250075 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250078 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250080 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250083 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250085 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250089 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250092 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250095 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250097 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250100 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250102 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250104 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250107 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:11.259111 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250109 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250113 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250116 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250120 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250123 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250125 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250128 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250130 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250134 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250137 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250140 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250143 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250146 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250149 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250152 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250155 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250157 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250160 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250162 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:11.259593 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250165 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250168 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250170 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250173 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250175 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250179 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250182 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250188 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250191 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250194 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250197 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250199 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250202 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250205 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250207 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250210 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250214 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250216 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250219 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250221 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:11.260071 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250224 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250226 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250229 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250231 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250233 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250236 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250239 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250242 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250245 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250248 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250251 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250253 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250255 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250258 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250260 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250262 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250265 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250269 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250272 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250274 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:11.260574 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250277 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250280 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.250282 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.250730 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.257628 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.257644 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.258378 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.258386 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258447 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258452 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258455 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258458 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258462 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258464 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258467 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:11.261060 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258470 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258474 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258476 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258479 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258482 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258484 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258487 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258490 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258492 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258495 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258497 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258500 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258502 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258505 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258507 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258510 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258513 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258516 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258518 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:11.261445 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258521 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258523 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258526 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258530 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258533 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258536 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258539 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258541 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258546 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258550 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258552 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258556 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258558 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258561 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258564 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258566 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258569 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258571 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258574 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258577 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:11.261898 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258580 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258582 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258585 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258588 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258590 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258593 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258596 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258598 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258601 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258603 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258606 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258608 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258611 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258613 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258616 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258619 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258622 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258625 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258628 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258631 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:11.262480 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258633 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258636 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258638 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258641 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258643 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258647 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258651 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258654 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258656 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258658 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258661 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258663 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258666 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258668 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258671 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258673 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258675 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258678 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258680 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258684 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:11.262975 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258686 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258689 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258691 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258694 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258696 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258700 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258703 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258706 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258709 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258713 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258715 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258718 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258720 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.258726 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:45:11.263477 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258842 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258847 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258850 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258853 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258856 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258858 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258861 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258863 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258866 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258868 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258871 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258873 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258876 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258879 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258881 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258883 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258886 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258888 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258891 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258894 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 16:45:11.263908 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258896 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258899 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258902 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258904 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258907 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258910 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258912 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258915 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258917 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258920 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258923 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258925 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258927 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258930 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258934 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258937 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258940 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258943 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258946 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 16:45:11.264415 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258948 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258951 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258953 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258956 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258959 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258962 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258964 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258967 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258971 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258974 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258977 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258980 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258983 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258986 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258989 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258991 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258994 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258997 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.258999 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 16:45:11.264875 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259001 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259005 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259008 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259011 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259014 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259016 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259019 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259021 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259023 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259026 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259028 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259030 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259033 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259035 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259038 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259040 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259043 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259045 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259048 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 16:45:11.265336 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259050 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259052 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259055 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259058 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259061 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259063 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259066 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259068 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259071 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259073 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259075 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259078 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259080 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259083 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259085 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259088 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259090 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259093 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259095 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 16:45:11.265807 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259098 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259100 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:11.259102 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.259107 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.259222 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.261109 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.262216 2579 server.go:1019] "Starting client certificate rotation" Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.262323 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.262389 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.264930 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 16:45:11.266283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.266089 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 16:45:11.276733 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.276704 2579 log.go:25] "Validated CRI v1 runtime API" Jul 20 16:45:11.278875 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.278858 2579 log.go:25] "Validated CRI v1 image API" Jul 20 16:45:11.279682 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.279666 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 16:45:11.282100 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.282080 2579 fs.go:135] Filesystem UUIDs: map[0535fcc8-872f-4863-ad0a-d6c207ad1e4b:/dev/nvme0n1p4 5fd7efed-d293-437d-921e-aa849a0a9cd5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 20 16:45:11.282158 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.282101 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 16:45:11.287169 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.287061 2579 manager.go:216] Machine: {Timestamp:2026-07-20 16:45:11.285763376 +0000 UTC m=+0.300185623 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100209 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec207e575bec79b5c91f639b847c275e SystemUUID:ec207e57-5bec-79b5-c91f-639b847c275e BootID:65ae73a2-9dc3-4975-a057-c38711e54341 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:12:be:f5:d0:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:12:be:f5:d0:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:69:75:e8:3c:fa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 16:45:11.287169 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.287166 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 16:45:11.287298 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.287281 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 16:45:11.289029 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.289006 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 16:45:11.289164 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.289032 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-206.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 16:45:11.289724 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.289713 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 16:45:11.289756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.289727 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 16:45:11.289756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.289735 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 16:45:11.290257 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.290247 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 16:45:11.290651 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.290633 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 16:45:11.291409 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.291398 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 16:45:11.294563 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294552 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 20 16:45:11.294672 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294663 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 16:45:11.294714 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294693 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 20 16:45:11.294714 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294702 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 16:45:11.294780 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294716 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 16:45:11.294780 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294726 2579 kubelet.go:398] "Adding apiserver pod source" Jul 20 16:45:11.294780 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.294735 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 16:45:11.295120 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.295109 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 16:45:11.295176 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.295168 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 16:45:11.297067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.297053 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 16:45:11.298467 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.298454 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 16:45:11.298521 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.298484 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 16:45:11.299044 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299034 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299049 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299055 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299060 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299068 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299073 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299081 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299086 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299092 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299097 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 16:45:11.299103 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299107 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 16:45:11.299368 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299112 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 16:45:11.299856 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299846 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 16:45:11.299856 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.299855 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 16:45:11.303003 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.302988 2579 server.go:1268] "Started kubelet" Jul 20 16:45:11.303176 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.303103 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 16:45:11.303245 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.303197 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 16:45:11.303245 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.303189 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 16:45:11.303864 ip-10-0-131-206 systemd[1]: Started Kubernetes Kubelet. Jul 20 16:45:11.304624 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.304502 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 20 16:45:11.308141 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.308113 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 16:45:11.309155 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.309130 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-206.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 16:45:11.309935 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.309901 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 16:45:11.310015 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.309902 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 16:45:11.310067 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.310015 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 16:45:11.313954 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.313935 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 16:45:11.313954 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.313951 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 16:45:11.314075 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.314000 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.314347 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.313335 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-206.ec2.internal.18c40d3dc57bf57f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-206.ec2.internal,UID:ip-10-0-131-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-206.ec2.internal,},FirstTimestamp:2026-07-20 16:45:11.302960511 +0000 UTC m=+0.317382758,LastTimestamp:2026-07-20 16:45:11.302960511 +0000 UTC m=+0.317382758,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-206.ec2.internal,}" Jul 20 16:45:11.314777 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.314729 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 16:45:11.315001 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.314976 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 16:45:11.315063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315015 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 16:45:11.315063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315027 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 16:45:11.315063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315042 2579 factory.go:55] Registering systemd factory Jul 20 16:45:11.315063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315051 2579 factory.go:223] Registration of the systemd container factory successfully Jul 20 16:45:11.315221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315130 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 20 16:45:11.315221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315139 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 20 16:45:11.315512 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315496 2579 factory.go:153] Registering CRI-O factory Jul 20 16:45:11.315575 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315516 2579 factory.go:223] Registration of the crio container factory successfully Jul 20 16:45:11.315575 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315542 2579 factory.go:103] Registering Raw factory Jul 20 16:45:11.315575 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.315556 2579 manager.go:1196] Started watching for new ooms in manager Jul 20 16:45:11.316160 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.316144 2579 manager.go:349] Starting recovery of all containers Jul 20 16:45:11.319512 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.319483 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 16:45:11.319601 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.319537 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 16:45:11.328446 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.328430 2579 manager.go:354] Recovery completed Jul 20 16:45:11.333537 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.333524 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.335418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.335391 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.335511 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.335435 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.335511 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.335449 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.336011 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.335996 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 16:45:11.336011 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.336009 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 16:45:11.336132 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.336025 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 20 16:45:11.336132 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.336065 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.336835 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.336816 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.336904 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.336849 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.336904 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.336870 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.337327 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.337299 2579 policy_none.go:49] "None policy: Start" Jul 20 16:45:11.337401 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.337332 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 16:45:11.337401 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.337346 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 16:45:11.337492 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.337345 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-206.ec2.internal.18c40d3dc76b38e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-206.ec2.internal,UID:ip-10-0-131-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-206.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-206.ec2.internal,},FirstTimestamp:2026-07-20 16:45:11.335418088 +0000 UTC m=+0.349840341,LastTimestamp:2026-07-20 16:45:11.335418088 +0000 UTC m=+0.349840341,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-206.ec2.internal,}" Jul 20 16:45:11.338485 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.338472 2579 policy_none.go:47] "Start" Jul 20 16:45:11.346947 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.346928 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lmlx5" Jul 20 16:45:11.354261 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.354244 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lmlx5" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.383525 2579 manager.go:337] "Starting Device Plugin manager" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.383740 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.383764 2579 server.go:87] "Starting device plugin registration server" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.384018 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.384030 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.384147 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.384260 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.384270 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.384945 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 16:45:11.396026 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.384999 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.466807 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.466764 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 16:45:11.468174 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.468153 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 16:45:11.468322 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.468194 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 16:45:11.468322 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.468229 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 16:45:11.468322 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.468265 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 16:45:11.470267 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.470246 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:11.484415 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.484396 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.486524 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.486507 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.486595 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.486536 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.486595 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.486546 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.486595 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.486568 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.495334 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.495281 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.495334 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.495318 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-206.ec2.internal\": node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.551022 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.550995 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.569071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.569033 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal"] Jul 20 16:45:11.569203 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.569128 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.570079 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.570054 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.570176 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.570093 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.570176 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.570106 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.571393 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.571378 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.571518 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.571502 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.571576 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.571534 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.572130 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.572116 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.572204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.572129 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.572204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.572142 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.572204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.572149 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.572204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.572154 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.572204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.572160 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.573289 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.573276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.573359 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.573301 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:11.573936 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.573920 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:11.574004 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.573952 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:11.574004 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.573967 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:11.606191 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.606159 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-206.ec2.internal\" not found" node="ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.610672 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.610653 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-206.ec2.internal\" not found" node="ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.617364 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.617343 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a82b47fcca87ff0c2367beb496455e0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal\" (UID: \"4a82b47fcca87ff0c2367beb496455e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.617456 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.617369 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a82b47fcca87ff0c2367beb496455e0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal\" (UID: \"4a82b47fcca87ff0c2367beb496455e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.617456 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.617389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/53461b03e56e0494bd965b8a7e0d36e4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-206.ec2.internal\" (UID: \"53461b03e56e0494bd965b8a7e0d36e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.651069 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.651044 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.717999 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.717967 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a82b47fcca87ff0c2367beb496455e0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal\" (UID: \"4a82b47fcca87ff0c2367beb496455e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.717999 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.717986 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a82b47fcca87ff0c2367beb496455e0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal\" (UID: \"4a82b47fcca87ff0c2367beb496455e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.718121 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.718016 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a82b47fcca87ff0c2367beb496455e0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal\" (UID: \"4a82b47fcca87ff0c2367beb496455e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.718121 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.718033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/53461b03e56e0494bd965b8a7e0d36e4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-206.ec2.internal\" (UID: \"53461b03e56e0494bd965b8a7e0d36e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.718121 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.718061 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a82b47fcca87ff0c2367beb496455e0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal\" (UID: \"4a82b47fcca87ff0c2367beb496455e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.718121 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.718069 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/53461b03e56e0494bd965b8a7e0d36e4-config\") pod \"kube-apiserver-proxy-ip-10-0-131-206.ec2.internal\" (UID: \"53461b03e56e0494bd965b8a7e0d36e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.752164 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.752084 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.853018 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.852981 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:11.908207 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.908175 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.913954 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:11.913935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" Jul 20 16:45:11.954016 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:11.953989 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.054620 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:12.054517 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.140733 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.140705 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:12.155329 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:12.155283 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.255931 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:12.255882 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.262052 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.262031 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 16:45:12.262205 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.262187 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:45:12.262267 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.262219 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 16:45:12.357036 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:12.356952 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.359123 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.359076 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 16:40:11 +0000 UTC" deadline="2028-01-11 12:38:46.211684132 +0000 UTC" Jul 20 16:45:12.359207 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.359126 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12955h53m33.85256321s" Jul 20 16:45:12.457840 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:12.457806 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.458925 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.458897 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 16:45:12.460133 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.460114 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientMemory" Jul 20 16:45:12.460206 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.460147 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 16:45:12.460206 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.460157 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeHasSufficientPID" Jul 20 16:45:12.558122 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:12.558093 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-206.ec2.internal\" not found" Jul 20 16:45:12.558546 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:12.558510 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a82b47fcca87ff0c2367beb496455e0.slice/crio-c7983fa626a39284f16da7e9a7cc6f98d82aa6784c953dac19a2e36125812a1b WatchSource:0}: Error finding container c7983fa626a39284f16da7e9a7cc6f98d82aa6784c953dac19a2e36125812a1b: Status 404 returned error can't find the container with id c7983fa626a39284f16da7e9a7cc6f98d82aa6784c953dac19a2e36125812a1b Jul 20 16:45:12.559655 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:12.559635 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53461b03e56e0494bd965b8a7e0d36e4.slice/crio-27f4abbd52e1652c68be2427e1362f35c1547a97d7af0f67cc18fed4975fccf3 WatchSource:0}: Error finding container 27f4abbd52e1652c68be2427e1362f35c1547a97d7af0f67cc18fed4975fccf3: Status 404 returned error can't find the container with id 27f4abbd52e1652c68be2427e1362f35c1547a97d7af0f67cc18fed4975fccf3 Jul 20 16:45:12.564076 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.564061 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:45:12.623734 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.623641 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:12.714510 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.714477 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" Jul 20 16:45:12.725253 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.725227 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 16:45:12.726013 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.726001 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" Jul 20 16:45:12.737913 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.737899 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 16:45:12.747571 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:12.747555 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:13.296504 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.296423 2579 apiserver.go:52] "Watching apiserver" Jul 20 16:45:13.312158 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.312102 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 16:45:13.315122 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.314587 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-zzjt5","kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm","openshift-cluster-node-tuning-operator/tuned-t9wkk","openshift-dns/node-resolver-krvrr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal","openshift-multus/multus-additional-cni-plugins-9xmgj","openshift-multus/multus-d7b9p","kube-system/global-pull-secret-syncer-4lj79","openshift-image-registry/node-ca-988qn","openshift-multus/network-metrics-daemon-r7zpb","openshift-network-diagnostics/network-check-target-l2prr","openshift-network-operator/iptables-alerter-kghz6","openshift-ovn-kubernetes/ovnkube-node-wmxlb"] Jul 20 16:45:13.315122 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.314599 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 16:45:13.317287 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.317264 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.319045 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.318669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.320080 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.319694 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.322134 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.320768 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.322134 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.321588 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8ghx2\"" Jul 20 16:45:13.322134 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.321632 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.322134 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.321782 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 16:45:13.322134 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.321920 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 16:45:13.322134 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.321954 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 16:45:13.322542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.322189 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.323107 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.322761 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.323759 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.324025 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.324329 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.324569 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhg7p\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.324864 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.325101 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.325336 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.325565 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s8n4p\"" Jul 20 16:45:13.325988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.325791 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.326662 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.326829 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.326988 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.327043 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.327054 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.327165 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nvr7h\"" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.327217 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f75sm\"" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.327544 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.329165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.327895 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-cnibin\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329255 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-os-release\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329283 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329312 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-system-cni-dir\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329466 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdk7r\" (UniqueName: \"kubernetes.io/projected/3fa635bd-8080-48a0-814d-9b794456bbe0-kube-api-access-jdk7r\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.329896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.329562 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.330888 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.330654 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.331999 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.331762 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 16:45:13.331999 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.331761 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 16:45:13.331999 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.331767 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.333545 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333527 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.333697 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333683 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-j6745\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333835 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333874 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333891 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333902 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.333902 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.334003 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.334057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.334101 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.334160 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tv7gn\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.334168 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-npdc4\"" Jul 20 16:45:13.334361 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.334264 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:13.334918 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.334612 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:13.336318 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.335251 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:13.336318 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.336026 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 16:45:13.337411 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.337389 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w2gzp\"" Jul 20 16:45:13.339745 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.339699 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:13.359621 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.359595 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnvcw" Jul 20 16:45:13.367860 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.367835 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnvcw" Jul 20 16:45:13.416265 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.416239 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 16:45:13.429921 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.429873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-env-overrides\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.429920 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-cni-multus\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.429953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtr6h\" (UniqueName: \"kubernetes.io/projected/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-kube-api-access-jtr6h\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.429980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8e062be-3e22-4169-ae03-1877d52568ff-konnectivity-ca\") pod \"konnectivity-agent-zzjt5\" (UID: \"e8e062be-3e22-4169-ae03-1877d52568ff\") " pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430004 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-host\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430025 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-systemd-units\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-run-netns\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430072 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-etc-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-log-socket\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-os-release\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430126 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430154 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-systemd\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.430213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430269 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-sys\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430315 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a434ee-9394-44d3-bfa8-de9ee981cc63-tmp-dir\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430339 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-cni-bin\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430384 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-kubelet\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430493 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42960ad8-01d5-4559-b2f0-956413454cca-tmp\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430607 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-system-cni-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430648 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-cnibin\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430677 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-socket-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430737 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430802 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-lib-modules\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430828 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42960ad8-01d5-4559-b2f0-956413454cca-etc-tuned\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.431067 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430853 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmpf8\" (UniqueName: \"kubernetes.io/projected/15a867d1-5903-4d89-96da-868c00db9f62-kube-api-access-nmpf8\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430879 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-netns\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87vzx\" (UniqueName: \"kubernetes.io/projected/a6a434ee-9394-44d3-bfa8-de9ee981cc63-kube-api-access-87vzx\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430929 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-ovn\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430953 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-cni-binary-copy\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.430979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-serviceca\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431052 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-systemd\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/15a867d1-5903-4d89-96da-868c00db9f62-ovn-node-metrics-cert\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-socket-dir-parent\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99f6f384-ef97-4d7c-8931-0547db955549-host-slash\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431182 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431222 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-kubernetes\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431255 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-cnibin\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dhjx\" (UniqueName: \"kubernetes.io/projected/99f6f384-ef97-4d7c-8931-0547db955549-kube-api-access-6dhjx\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-device-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431333 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-sys-fs\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.431866 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431359 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8e062be-3e22-4169-ae03-1877d52568ff-agent-certs\") pod \"konnectivity-agent-zzjt5\" (UID: \"e8e062be-3e22-4169-ae03-1877d52568ff\") " pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431414 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431094 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-var-lib-kubelet\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431568 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431578 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-kubelet\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-k8s-cni-cncf-io\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431642 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-conf-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431669 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-etc-kubernetes\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431695 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9pd\" (UniqueName: \"kubernetes.io/projected/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-kube-api-access-4r9pd\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdk7r\" (UniqueName: \"kubernetes.io/projected/3fa635bd-8080-48a0-814d-9b794456bbe0-kube-api-access-jdk7r\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431732 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-hostroot\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431780 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-daemon-config\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99f6f384-ef97-4d7c-8931-0547db955549-iptables-alerter-script\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431819 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-cnibin\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.432637 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/416d3c60-3d18-4f9e-a38f-4e5b085467d9-kubelet-config\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431928 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-os-release\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431961 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-modprobe-d\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.431994 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-node-log\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432024 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432053 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-host\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-system-cni-dir\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a6a434ee-9394-44d3-bfa8-de9ee981cc63-hosts-file\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432179 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-system-cni-dir\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-cni-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-multus-certs\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432283 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsqmh\" (UniqueName: \"kubernetes.io/projected/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-kube-api-access-gsqmh\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432321 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fa635bd-8080-48a0-814d-9b794456bbe0-os-release\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432331 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysctl-d\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysctl-conf\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432412 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-run\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll8rq\" (UniqueName: \"kubernetes.io/projected/42960ad8-01d5-4559-b2f0-956413454cca-kube-api-access-ll8rq\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.433491 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-slash\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-ovnkube-script-lib\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-registration-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432680 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysconfig\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-cni-netd\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432886 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-ovnkube-config\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432925 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-cni-bin\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.432970 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbbrb\" (UniqueName: \"kubernetes.io/projected/485b36f3-7817-4692-8148-43c61858be1b-kube-api-access-jbbrb\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.433003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/416d3c60-3d18-4f9e-a38f-4e5b085467d9-dbus\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.433029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-var-lib-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.434454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.433123 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fa635bd-8080-48a0-814d-9b794456bbe0-cni-binary-copy\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.442161 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.442127 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 16:45:13.447799 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.447766 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdk7r\" (UniqueName: \"kubernetes.io/projected/3fa635bd-8080-48a0-814d-9b794456bbe0-kube-api-access-jdk7r\") pod \"multus-additional-cni-plugins-9xmgj\" (UID: \"3fa635bd-8080-48a0-814d-9b794456bbe0\") " pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.475691 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.475627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" event={"ID":"53461b03e56e0494bd965b8a7e0d36e4","Type":"ContainerStarted","Data":"27f4abbd52e1652c68be2427e1362f35c1547a97d7af0f67cc18fed4975fccf3"} Jul 20 16:45:13.478140 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.478105 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" event={"ID":"4a82b47fcca87ff0c2367beb496455e0","Type":"ContainerStarted","Data":"c7983fa626a39284f16da7e9a7cc6f98d82aa6784c953dac19a2e36125812a1b"} Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-log-socket\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-os-release\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-systemd\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533535 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-log-socket\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533598 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533630 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-sys\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a434ee-9394-44d3-bfa8-de9ee981cc63-tmp-dir\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533705 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-cni-bin\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-kubelet\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533741 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-sys\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533761 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42960ad8-01d5-4559-b2f0-956413454cca-tmp\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.534403 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533808 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-system-cni-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-cnibin\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533856 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-socket-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533863 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-cni-bin\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533662 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-os-release\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533883 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.533981 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.534066 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:14.034043888 +0000 UTC m=+3.048466140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534235 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a434ee-9394-44d3-bfa8-de9ee981cc63-tmp-dir\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.533688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-systemd\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534298 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-system-cni-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-lib-modules\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534385 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-socket-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42960ad8-01d5-4559-b2f0-956413454cca-etc-tuned\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmpf8\" (UniqueName: \"kubernetes.io/projected/15a867d1-5903-4d89-96da-868c00db9f62-kube-api-access-nmpf8\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534536 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534577 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-kubelet\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.535460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-netns\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534680 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-lib-modules\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87vzx\" (UniqueName: \"kubernetes.io/projected/a6a434ee-9394-44d3-bfa8-de9ee981cc63-kube-api-access-87vzx\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534785 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-cnibin\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.534817 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-ovn\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-cni-binary-copy\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.534881 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:14.034863346 +0000 UTC m=+3.049285609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-serviceca\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-systemd\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/15a867d1-5903-4d89-96da-868c00db9f62-ovn-node-metrics-cert\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.534987 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-socket-dir-parent\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99f6f384-ef97-4d7c-8931-0547db955549-host-slash\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-kubernetes\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dhjx\" (UniqueName: \"kubernetes.io/projected/99f6f384-ef97-4d7c-8931-0547db955549-kube-api-access-6dhjx\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535150 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-device-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.536313 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-sys-fs\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535198 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8e062be-3e22-4169-ae03-1877d52568ff-agent-certs\") pod \"konnectivity-agent-zzjt5\" (UID: \"e8e062be-3e22-4169-ae03-1877d52568ff\") " pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535229 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-var-lib-kubelet\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535357 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-cni-binary-copy\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535911 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99f6f384-ef97-4d7c-8931-0547db955549-host-slash\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.535978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-kubelet\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-k8s-cni-cncf-io\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-conf-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-etc-kubernetes\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9pd\" (UniqueName: \"kubernetes.io/projected/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-kube-api-access-4r9pd\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-hostroot\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-daemon-config\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536215 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99f6f384-ef97-4d7c-8931-0547db955549-iptables-alerter-script\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/416d3c60-3d18-4f9e-a38f-4e5b085467d9-kubelet-config\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536268 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-modprobe-d\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-node-log\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536393 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536420 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-host\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.536997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536446 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a6a434ee-9394-44d3-bfa8-de9ee981cc63-hosts-file\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536471 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-cni-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-multus-certs\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsqmh\" (UniqueName: \"kubernetes.io/projected/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-kube-api-access-gsqmh\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536560 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysctl-d\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysctl-conf\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-run\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536635 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll8rq\" (UniqueName: \"kubernetes.io/projected/42960ad8-01d5-4559-b2f0-956413454cca-kube-api-access-ll8rq\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-slash\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536687 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-ovnkube-script-lib\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536714 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-registration-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysconfig\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-cni-netd\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-ovnkube-config\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-cni-bin\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536858 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbbrb\" (UniqueName: \"kubernetes.io/projected/485b36f3-7817-4692-8148-43c61858be1b-kube-api-access-jbbrb\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.536989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-cni-bin\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.537765 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/416d3c60-3d18-4f9e-a38f-4e5b085467d9-dbus\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-var-lib-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537070 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-env-overrides\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-cni-multus\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537126 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtr6h\" (UniqueName: \"kubernetes.io/projected/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-kube-api-access-jtr6h\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537152 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8e062be-3e22-4169-ae03-1877d52568ff-konnectivity-ca\") pod \"konnectivity-agent-zzjt5\" (UID: \"e8e062be-3e22-4169-ae03-1877d52568ff\") " pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-host\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537182 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-netns\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-systemd-units\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537230 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-run-netns\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537261 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-etc-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-etc-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537511 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysctl-d\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537637 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysctl-conf\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537680 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-modprobe-d\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537724 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-var-lib-kubelet\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537737 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-run\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537762 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-kubelet\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.538268 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537798 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-k8s-cni-cncf-io\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537833 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-conf-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537866 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-etc-kubernetes\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.537956 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-slash\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.538049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-hostroot\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.538464 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-ovn\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.538531 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-daemon-config\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.538914 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-serviceca\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.538970 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-run-systemd\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539019 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-host\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539062 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/416d3c60-3d18-4f9e-a38f-4e5b085467d9-kubelet-config\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539093 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-node-log\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539019 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99f6f384-ef97-4d7c-8931-0547db955549-iptables-alerter-script\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539171 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-cni-dir\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539215 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-registration-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-sysconfig\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-cni-netd\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539430 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/416d3c60-3d18-4f9e-a38f-4e5b085467d9-dbus\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539484 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-run-multus-certs\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539538 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a6a434ee-9394-44d3-bfa8-de9ee981cc63-hosts-file\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539569 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-env-overrides\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-etc-kubernetes\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539628 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-multus-socket-dir-parent\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539637 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539726 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-var-lib-openvswitch\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.539809 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539768 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-device-dir\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.540440 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539919 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-systemd-units\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.540440 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539927 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/485b36f3-7817-4692-8148-43c61858be1b-sys-fs\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.540440 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.539963 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/42960ad8-01d5-4559-b2f0-956413454cca-host\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.540440 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.540001 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-host-var-lib-cni-multus\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.540440 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.540043 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/15a867d1-5903-4d89-96da-868c00db9f62-host-run-netns\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.540740 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.540471 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8e062be-3e22-4169-ae03-1877d52568ff-konnectivity-ca\") pod \"konnectivity-agent-zzjt5\" (UID: \"e8e062be-3e22-4169-ae03-1877d52568ff\") " pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.540740 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.540676 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42960ad8-01d5-4559-b2f0-956413454cca-tmp\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.540915 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.540878 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8e062be-3e22-4169-ae03-1877d52568ff-agent-certs\") pod \"konnectivity-agent-zzjt5\" (UID: \"e8e062be-3e22-4169-ae03-1877d52568ff\") " pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.541479 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.541046 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-ovnkube-config\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.541479 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.541219 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/15a867d1-5903-4d89-96da-868c00db9f62-ovn-node-metrics-cert\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.541479 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.541443 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/15a867d1-5903-4d89-96da-868c00db9f62-ovnkube-script-lib\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.542613 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.542575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/42960ad8-01d5-4559-b2f0-956413454cca-etc-tuned\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.547441 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.547359 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:13.547441 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.547405 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:13.547441 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.547419 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:13.547630 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:13.547494 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:14.047473428 +0000 UTC m=+3.061895663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:13.553127 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.553096 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmpf8\" (UniqueName: \"kubernetes.io/projected/15a867d1-5903-4d89-96da-868c00db9f62-kube-api-access-nmpf8\") pod \"ovnkube-node-wmxlb\" (UID: \"15a867d1-5903-4d89-96da-868c00db9f62\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.553954 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.553582 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9pd\" (UniqueName: \"kubernetes.io/projected/d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8-kube-api-access-4r9pd\") pod \"multus-d7b9p\" (UID: \"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8\") " pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.555081 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.554764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbbrb\" (UniqueName: \"kubernetes.io/projected/485b36f3-7817-4692-8148-43c61858be1b-kube-api-access-jbbrb\") pod \"aws-ebs-csi-driver-node-jnqzm\" (UID: \"485b36f3-7817-4692-8148-43c61858be1b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.556312 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.556142 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtr6h\" (UniqueName: \"kubernetes.io/projected/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-kube-api-access-jtr6h\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:13.556312 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.556275 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dhjx\" (UniqueName: \"kubernetes.io/projected/99f6f384-ef97-4d7c-8931-0547db955549-kube-api-access-6dhjx\") pod \"iptables-alerter-kghz6\" (UID: \"99f6f384-ef97-4d7c-8931-0547db955549\") " pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.556996 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.556952 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll8rq\" (UniqueName: \"kubernetes.io/projected/42960ad8-01d5-4559-b2f0-956413454cca-kube-api-access-ll8rq\") pod \"tuned-t9wkk\" (UID: \"42960ad8-01d5-4559-b2f0-956413454cca\") " pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.558563 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.558358 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87vzx\" (UniqueName: \"kubernetes.io/projected/a6a434ee-9394-44d3-bfa8-de9ee981cc63-kube-api-access-87vzx\") pod \"node-resolver-krvrr\" (UID: \"a6a434ee-9394-44d3-bfa8-de9ee981cc63\") " pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.558685 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.558660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsqmh\" (UniqueName: \"kubernetes.io/projected/d06a057d-92d0-43c3-bd8c-487f8ab1e22b-kube-api-access-gsqmh\") pod \"node-ca-988qn\" (UID: \"d06a057d-92d0-43c3-bd8c-487f8ab1e22b\") " pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:13.568272 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.568248 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:13.637076 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.637040 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" Jul 20 16:45:13.648756 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:13.648720 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa635bd_8080_48a0_814d_9b794456bbe0.slice/crio-57e4a399947cacf210c8a6ff40d1554992a9dc7af0fe172a9aa645687ffdf87b WatchSource:0}: Error finding container 57e4a399947cacf210c8a6ff40d1554992a9dc7af0fe172a9aa645687ffdf87b: Status 404 returned error can't find the container with id 57e4a399947cacf210c8a6ff40d1554992a9dc7af0fe172a9aa645687ffdf87b Jul 20 16:45:13.649537 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.649518 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kghz6" Jul 20 16:45:13.658286 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:13.658254 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f6f384_ef97_4d7c_8931_0547db955549.slice/crio-4e6953d9cd1b0a5931339f8931aa4b6c46841f11e1e1a7dcdf757fbdb6452983 WatchSource:0}: Error finding container 4e6953d9cd1b0a5931339f8931aa4b6c46841f11e1e1a7dcdf757fbdb6452983: Status 404 returned error can't find the container with id 4e6953d9cd1b0a5931339f8931aa4b6c46841f11e1e1a7dcdf757fbdb6452983 Jul 20 16:45:13.659246 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.659221 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" Jul 20 16:45:13.667981 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.667958 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" Jul 20 16:45:13.668317 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:13.668287 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485b36f3_7817_4692_8148_43c61858be1b.slice/crio-283a0f0f092caafd6429d7908c89c27d97774f13014bc3fcbbaa6d79e89aaa04 WatchSource:0}: Error finding container 283a0f0f092caafd6429d7908c89c27d97774f13014bc3fcbbaa6d79e89aaa04: Status 404 returned error can't find the container with id 283a0f0f092caafd6429d7908c89c27d97774f13014bc3fcbbaa6d79e89aaa04 Jul 20 16:45:13.678221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.678170 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krvrr" Jul 20 16:45:13.687006 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.686969 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:13.697954 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.697929 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d7b9p" Jul 20 16:45:13.706942 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.706922 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:13.714594 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:13.714572 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-988qn" Jul 20 16:45:14.031237 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:14.031138 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42960ad8_01d5_4559_b2f0_956413454cca.slice/crio-d03a5e343b77594af90286a799b433a85ceaf21ade9e909dac0ade1047c0d255 WatchSource:0}: Error finding container d03a5e343b77594af90286a799b433a85ceaf21ade9e909dac0ade1047c0d255: Status 404 returned error can't find the container with id d03a5e343b77594af90286a799b433a85ceaf21ade9e909dac0ade1047c0d255 Jul 20 16:45:14.041173 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.041147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:14.041296 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.041186 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:14.041357 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.041297 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:14.041357 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.041328 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:14.041477 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.041369 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:15.041353818 +0000 UTC m=+4.055776057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:14.041477 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.041403 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:15.041386087 +0000 UTC m=+4.055808322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:14.065025 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.064992 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 16:45:14.142229 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.142190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:14.142427 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.142403 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:14.142495 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.142433 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:14.142495 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.142448 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:14.142568 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.142516 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:15.142497656 +0000 UTC m=+4.156919901 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:14.372791 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.372661 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:40:13 +0000 UTC" deadline="2028-03-30 03:53:35.74014763 +0000 UTC" Jul 20 16:45:14.372791 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.372736 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14843h8m21.367421515s" Jul 20 16:45:14.468946 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.468904 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:14.469159 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:14.469047 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:14.481325 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.481278 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" event={"ID":"42960ad8-01d5-4559-b2f0-956413454cca","Type":"ContainerStarted","Data":"d03a5e343b77594af90286a799b433a85ceaf21ade9e909dac0ade1047c0d255"} Jul 20 16:45:14.482832 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.482799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" event={"ID":"485b36f3-7817-4692-8148-43c61858be1b","Type":"ContainerStarted","Data":"283a0f0f092caafd6429d7908c89c27d97774f13014bc3fcbbaa6d79e89aaa04"} Jul 20 16:45:14.484131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.484097 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kghz6" event={"ID":"99f6f384-ef97-4d7c-8931-0547db955549","Type":"ContainerStarted","Data":"4e6953d9cd1b0a5931339f8931aa4b6c46841f11e1e1a7dcdf757fbdb6452983"} Jul 20 16:45:14.485390 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:14.485361 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerStarted","Data":"57e4a399947cacf210c8a6ff40d1554992a9dc7af0fe172a9aa645687ffdf87b"} Jul 20 16:45:14.742458 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:14.742423 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a7d2b3_54b3_4587_a90a_6cce1a20b2f8.slice/crio-4b25dab18dc62857b376cb2e2de68e1c9e3444de67d57a4c6c8bb565a58715a4 WatchSource:0}: Error finding container 4b25dab18dc62857b376cb2e2de68e1c9e3444de67d57a4c6c8bb565a58715a4: Status 404 returned error can't find the container with id 4b25dab18dc62857b376cb2e2de68e1c9e3444de67d57a4c6c8bb565a58715a4 Jul 20 16:45:14.744613 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:14.744540 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a434ee_9394_44d3_bfa8_de9ee981cc63.slice/crio-5190b45e576654e80e93b4ad908aaaa68ef05fe64c13404ebfbb5d746ec1db6d WatchSource:0}: Error finding container 5190b45e576654e80e93b4ad908aaaa68ef05fe64c13404ebfbb5d746ec1db6d: Status 404 returned error can't find the container with id 5190b45e576654e80e93b4ad908aaaa68ef05fe64c13404ebfbb5d746ec1db6d Jul 20 16:45:14.751235 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:14.751207 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e062be_3e22_4169_ae03_1877d52568ff.slice/crio-6fa62336f2213f1b876c0f0ef0c0e6e517a762d2ca3f8c7e9d234f5723de887e WatchSource:0}: Error finding container 6fa62336f2213f1b876c0f0ef0c0e6e517a762d2ca3f8c7e9d234f5723de887e: Status 404 returned error can't find the container with id 6fa62336f2213f1b876c0f0ef0c0e6e517a762d2ca3f8c7e9d234f5723de887e Jul 20 16:45:14.752271 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:14.752244 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a867d1_5903_4d89_96da_868c00db9f62.slice/crio-26eea5fa3c21c7ca5935a61e42a564d943d64de05231249787e85faf1f72857c WatchSource:0}: Error finding container 26eea5fa3c21c7ca5935a61e42a564d943d64de05231249787e85faf1f72857c: Status 404 returned error can't find the container with id 26eea5fa3c21c7ca5935a61e42a564d943d64de05231249787e85faf1f72857c Jul 20 16:45:14.753448 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:14.753422 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06a057d_92d0_43c3_bd8c_487f8ab1e22b.slice/crio-0f6db001860a018941e6a60372a586020d3415843b13b6e336d7ed43e4dc7d3b WatchSource:0}: Error finding container 0f6db001860a018941e6a60372a586020d3415843b13b6e336d7ed43e4dc7d3b: Status 404 returned error can't find the container with id 0f6db001860a018941e6a60372a586020d3415843b13b6e336d7ed43e4dc7d3b Jul 20 16:45:15.051771 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.051731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:15.051918 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.051844 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:15.051994 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.051972 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:15.052052 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.052041 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:17.052018721 +0000 UTC m=+6.066440982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:15.052563 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.052539 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:15.052686 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.052602 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:17.052585916 +0000 UTC m=+6.067008170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:15.152919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.152870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:15.153115 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.153066 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:15.153115 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.153092 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:15.153115 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.153105 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:15.153280 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.153168 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:17.153146416 +0000 UTC m=+6.167568671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:15.373388 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.373251 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:40:13 +0000 UTC" deadline="2028-04-24 15:38:01.560719204 +0000 UTC" Jul 20 16:45:15.373388 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.373335 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15454h52m46.187389752s" Jul 20 16:45:15.469402 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.469367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:15.469601 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.469539 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:15.469669 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.469533 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:15.469669 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:15.469624 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:15.490534 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.490451 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-988qn" event={"ID":"d06a057d-92d0-43c3-bd8c-487f8ab1e22b","Type":"ContainerStarted","Data":"0f6db001860a018941e6a60372a586020d3415843b13b6e336d7ed43e4dc7d3b"} Jul 20 16:45:15.495008 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.494900 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"26eea5fa3c21c7ca5935a61e42a564d943d64de05231249787e85faf1f72857c"} Jul 20 16:45:15.497131 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.497059 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7b9p" event={"ID":"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8","Type":"ContainerStarted","Data":"4b25dab18dc62857b376cb2e2de68e1c9e3444de67d57a4c6c8bb565a58715a4"} Jul 20 16:45:15.502149 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.501501 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" event={"ID":"53461b03e56e0494bd965b8a7e0d36e4","Type":"ContainerStarted","Data":"2bc859492189ee7592d5829821de68ed95de0e5e9f0d0aceece6ae017855ce45"} Jul 20 16:45:15.508099 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.508053 2579 generic.go:356] "Generic (PLEG): container finished" podID="4a82b47fcca87ff0c2367beb496455e0" containerID="986f562d66115b17c154bce1cf491eb5df05eb38f264d9015f03bd0ef374baf4" exitCode=0 Jul 20 16:45:15.508227 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.508138 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" event={"ID":"4a82b47fcca87ff0c2367beb496455e0","Type":"ContainerDied","Data":"986f562d66115b17c154bce1cf491eb5df05eb38f264d9015f03bd0ef374baf4"} Jul 20 16:45:15.511064 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.511036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zzjt5" event={"ID":"e8e062be-3e22-4169-ae03-1877d52568ff","Type":"ContainerStarted","Data":"6fa62336f2213f1b876c0f0ef0c0e6e517a762d2ca3f8c7e9d234f5723de887e"} Jul 20 16:45:15.516759 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.516477 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krvrr" event={"ID":"a6a434ee-9394-44d3-bfa8-de9ee981cc63","Type":"ContainerStarted","Data":"5190b45e576654e80e93b4ad908aaaa68ef05fe64c13404ebfbb5d746ec1db6d"} Jul 20 16:45:15.520101 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:15.519634 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-206.ec2.internal" podStartSLOduration=2.519614406 podStartE2EDuration="2.519614406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:15.518485689 +0000 UTC m=+4.532907962" watchObservedRunningTime="2026-07-20 16:45:15.519614406 +0000 UTC m=+4.534036676" Jul 20 16:45:16.468498 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:16.468456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:16.469005 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:16.468610 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:16.541135 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:16.541091 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" event={"ID":"4a82b47fcca87ff0c2367beb496455e0","Type":"ContainerStarted","Data":"a40eb0f84fcd23fbf324e0bae407b0b5fc006e71c26d02e4f94bbc61970c2d4a"} Jul 20 16:45:17.072836 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:17.072001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:17.072836 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:17.072061 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:17.072836 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.072216 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:17.072836 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.072281 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:21.072262318 +0000 UTC m=+10.086684569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:17.072836 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.072743 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:17.072836 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.072796 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:21.072779329 +0000 UTC m=+10.087201567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:17.172847 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:17.172805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:17.173080 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.173059 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:17.173153 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.173089 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:17.173153 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.173102 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:17.173269 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.173163 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:21.173143638 +0000 UTC m=+10.187565892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:17.471709 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:17.471619 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:17.472217 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.471910 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:17.472333 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:17.472312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:17.472465 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:17.472443 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:18.469713 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:18.469667 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:18.469903 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:18.469812 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:19.469300 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:19.469266 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:19.469300 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:19.469318 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:19.469857 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:19.469456 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:19.469977 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:19.469949 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:20.469138 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:20.469059 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:20.469293 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:20.469191 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:21.106049 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:21.106015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:21.106049 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:21.106054 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:21.106527 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.106163 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:21.106527 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.106207 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:29.106195349 +0000 UTC m=+18.120617583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:21.106527 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.106508 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:21.106632 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.106541 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:29.10653097 +0000 UTC m=+18.120953204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:21.206920 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:21.206873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:21.207087 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.207053 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:21.207087 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.207073 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:21.207087 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.207085 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:21.207237 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.207141 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:29.207123292 +0000 UTC m=+18.221545541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:21.470076 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:21.469958 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:21.470076 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.470040 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:21.470335 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:21.470301 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:21.470418 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:21.470399 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:22.469425 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:22.469389 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:22.469896 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:22.469521 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:23.469145 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:23.469114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:23.469355 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:23.469235 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:23.469853 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:23.469687 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:23.469853 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:23.469808 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:24.468966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:24.468899 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:24.469225 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:24.469045 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:25.471345 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:25.471289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:25.471751 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:25.471440 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:25.471813 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:25.471796 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:25.471915 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:25.471889 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:26.469255 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:26.469220 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:26.469455 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:26.469378 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:27.468555 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:27.468518 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:27.469040 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:27.468671 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:27.469040 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:27.468738 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:27.469040 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:27.468872 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:28.468938 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:28.468899 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:28.469378 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:28.469051 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:29.172051 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:29.172017 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:29.172185 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:29.172067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:29.172235 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.172191 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:29.172299 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.172257 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:45.172238696 +0000 UTC m=+34.186660946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:29.172383 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.172295 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:29.172383 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.172356 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:45.17234078 +0000 UTC m=+34.186763015 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:29.272524 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:29.272466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:29.272704 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.272601 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:29.272704 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.272623 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:29.272704 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.272634 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:29.272704 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.272692 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:45.272675363 +0000 UTC m=+34.287097616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:29.469277 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:29.469238 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:29.469787 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.469402 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:29.469787 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:29.469462 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:29.469787 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:29.469576 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:30.469162 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:30.469123 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:30.469351 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:30.469251 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:31.472944 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:31.470706 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:31.472944 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:31.470992 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:31.472944 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:31.471446 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:31.472944 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:31.471573 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:32.469202 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:32.469165 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:32.469412 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:32.469295 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:33.469475 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:33.469440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:33.469894 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:33.469440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:33.469894 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:33.469564 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:33.469894 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:33.469612 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:34.469571 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.469384 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:34.470443 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:34.469645 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:34.581136 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.581041 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" event={"ID":"42960ad8-01d5-4559-b2f0-956413454cca","Type":"ContainerStarted","Data":"1f3fdada982d86974c1a6eb78de9cdf0320048a3fecb2e343b46abf8c3494c7a"} Jul 20 16:45:34.582541 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.582516 2579 generic.go:356] "Generic (PLEG): container finished" podID="3fa635bd-8080-48a0-814d-9b794456bbe0" containerID="9d3f49c69afba38f1e2e1a65e1891c366a006e3df327daaa924bbcd8e52f6b7a" exitCode=0 Jul 20 16:45:34.582678 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.582581 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerDied","Data":"9d3f49c69afba38f1e2e1a65e1891c366a006e3df327daaa924bbcd8e52f6b7a"} Jul 20 16:45:34.584229 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.584004 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-988qn" event={"ID":"d06a057d-92d0-43c3-bd8c-487f8ab1e22b","Type":"ContainerStarted","Data":"aae32c87e40eda36ce352f832ba37253c20ae630ab89acbb3643a46a1a9d4a1b"} Jul 20 16:45:34.586896 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.586872 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:45:34.587206 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587188 2579 generic.go:356] "Generic (PLEG): container finished" podID="15a867d1-5903-4d89-96da-868c00db9f62" containerID="6c97ba341b51653d4187f5cb248a8dcd0c2c9e299a039cddfec2a10e9829815b" exitCode=1 Jul 20 16:45:34.587282 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587253 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"2a0c3e928fe74b710e8f06dd3d90b61dc5e012d3b66da7a2320b29ba3b6da765"} Jul 20 16:45:34.587351 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587328 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"75ac3476f45510c9dfae2b006778e6ed902a318f18ad218b93aeec45a7b3114e"} Jul 20 16:45:34.587407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587377 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"c2eef4486a6849820db8fd18b14391679a2e2bfb42e0eb1dd351ee12cec7c06c"} Jul 20 16:45:34.587441 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"28891f0aa29ff37e45aaeed45957cf393071915c58297ef422f8d015929e879b"} Jul 20 16:45:34.587480 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587473 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerDied","Data":"6c97ba341b51653d4187f5cb248a8dcd0c2c9e299a039cddfec2a10e9829815b"} Jul 20 16:45:34.587530 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.587524 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"521930d00631c9a3edeeb81763f448d14762ec4f19b4847adbd53ece7a425c93"} Jul 20 16:45:34.588635 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.588609 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d7b9p" event={"ID":"d4a7d2b3-54b3-4587-a90a-6cce1a20b2f8","Type":"ContainerStarted","Data":"9ab224b8a52389986acf811dda792d1bcf5420c66c11f52329a8922958ca2a0b"} Jul 20 16:45:34.589860 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.589842 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" event={"ID":"485b36f3-7817-4692-8148-43c61858be1b","Type":"ContainerStarted","Data":"e9f2f349dd82dc86d22a904ca54f9bc1ad2faa86fe5a8141a290bb1d605e3d39"} Jul 20 16:45:34.591082 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.591060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zzjt5" event={"ID":"e8e062be-3e22-4169-ae03-1877d52568ff","Type":"ContainerStarted","Data":"0cf664b81c26d10a5febda3eb5e1f9960f5f2c8fd32a8c09edce948367f29f79"} Jul 20 16:45:34.592171 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.592151 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krvrr" event={"ID":"a6a434ee-9394-44d3-bfa8-de9ee981cc63","Type":"ContainerStarted","Data":"97b4c50980a542bf366328f8ab712c741bad4a12f71a58ad4c09b8d379f7c919"} Jul 20 16:45:34.598260 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.598221 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t9wkk" podStartSLOduration=3.006808309 podStartE2EDuration="22.598209324s" totalImagesPullingTime="19.591401015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:34.598010003 +0000 UTC m=+23.612432261" watchObservedRunningTime="2026-07-20 16:45:34.598209324 +0000 UTC m=+23.612631606" Jul 20 16:45:34.598425 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.598400 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-206.ec2.internal" podStartSLOduration=21.598394141 podStartE2EDuration="21.598394141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:16.558185848 +0000 UTC m=+5.572608108" watchObservedRunningTime="2026-07-20 16:45:34.598394141 +0000 UTC m=+23.612816398" Jul 20 16:45:34.645526 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.645476 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-krvrr" podStartSLOduration=3.770941617 podStartE2EDuration="22.645460572s" totalImagesPullingTime="18.874518955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:34.644909816 +0000 UTC m=+23.659332071" watchObservedRunningTime="2026-07-20 16:45:34.645460572 +0000 UTC m=+23.659882834" Jul 20 16:45:34.659525 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.659471 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-988qn" podStartSLOduration=3.79108166 podStartE2EDuration="22.65945461s" totalImagesPullingTime="18.86837295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:34.659401917 +0000 UTC m=+23.673824173" watchObservedRunningTime="2026-07-20 16:45:34.65945461 +0000 UTC m=+23.673876868" Jul 20 16:45:34.674444 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.674400 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zzjt5" podStartSLOduration=3.804403985 podStartE2EDuration="22.674386906s" totalImagesPullingTime="18.869982921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:34.674053707 +0000 UTC m=+23.688475963" watchObservedRunningTime="2026-07-20 16:45:34.674386906 +0000 UTC m=+23.688809161" Jul 20 16:45:34.692008 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:34.691957 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d7b9p" podStartSLOduration=3.767623598 podStartE2EDuration="22.69194369s" totalImagesPullingTime="18.924320092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:34.691649914 +0000 UTC m=+23.706072170" watchObservedRunningTime="2026-07-20 16:45:34.69194369 +0000 UTC m=+23.706365945" Jul 20 16:45:35.469174 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:35.469129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:35.469340 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:35.469139 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:35.469340 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:35.469241 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:35.469438 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:35.469363 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:35.489046 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:35.489012 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 16:45:35.596431 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:35.596392 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" event={"ID":"485b36f3-7817-4692-8148-43c61858be1b","Type":"ContainerStarted","Data":"a153037ba49d0c89d424a2be8f8d052d28324df428ee7cf6a42ebcc29622d3d8"} Jul 20 16:45:35.598015 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:35.597985 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kghz6" event={"ID":"99f6f384-ef97-4d7c-8931-0547db955549","Type":"ContainerStarted","Data":"fae9a5495c33bfe5562defd5a0fbd0b7049539fb17c60f12028f915f5268ab1f"} Jul 20 16:45:36.401874 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.401769 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T16:45:35.489039421Z","UUID":"6eaf2fdb-8a2b-46ed-bb25-bb3c35854dc6","Handler":null,"Name":"","Endpoint":""} Jul 20 16:45:36.403703 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.403442 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 16:45:36.403703 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.403507 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 16:45:36.468692 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.468667 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:36.468820 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:36.468791 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:36.603143 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.603104 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" event={"ID":"485b36f3-7817-4692-8148-43c61858be1b","Type":"ContainerStarted","Data":"36f2bae2b277366ab3dafdd74f187b1a6913ce31716d0300562fbf759c0ef161"} Jul 20 16:45:36.606635 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.606613 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:45:36.607005 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.606980 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"e24f91edd626e015c459afa1e24991ccb06e4342ddc563dfe7c907690a5e9883"} Jul 20 16:45:36.625214 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.625166 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnqzm" podStartSLOduration=2.258965381 podStartE2EDuration="24.625148702s" totalImagesPullingTime="22.366183321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:36.624702383 +0000 UTC m=+25.639124670" watchObservedRunningTime="2026-07-20 16:45:36.625148702 +0000 UTC m=+25.639570957" Jul 20 16:45:36.625840 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:36.625803 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kghz6" podStartSLOduration=4.677736015 podStartE2EDuration="24.625793991s" totalImagesPullingTime="19.948057976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:35.613560345 +0000 UTC m=+24.627982612" watchObservedRunningTime="2026-07-20 16:45:36.625793991 +0000 UTC m=+25.640216300" Jul 20 16:45:37.468824 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:37.468787 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:37.468824 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:37.468803 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:37.469076 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:37.468927 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:37.469128 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:37.469074 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:38.354494 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:38.354287 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:38.355021 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:38.355005 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:38.468912 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:38.468871 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:38.469139 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:38.468998 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:38.612431 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:38.612355 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:38.613014 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:38.612984 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zzjt5" Jul 20 16:45:39.469226 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:39.469196 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:39.469856 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:39.469198 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:39.469856 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:39.469455 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:39.470128 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:39.470107 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:40.469595 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:40.469413 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:40.470202 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:40.469665 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:40.618458 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:40.618426 2579 generic.go:356] "Generic (PLEG): container finished" podID="3fa635bd-8080-48a0-814d-9b794456bbe0" containerID="16037240e8eceec5c4553787d3ef78e5ef236d22a6513b41bc4e6c3db252cbdd" exitCode=0 Jul 20 16:45:40.618649 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:40.618509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerDied","Data":"16037240e8eceec5c4553787d3ef78e5ef236d22a6513b41bc4e6c3db252cbdd"} Jul 20 16:45:40.622132 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:40.622111 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:45:40.624185 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:40.624156 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"98f74b39c72ca18f321450b88bdf32a6bba094fc690624d16723ca7575f6d898"} Jul 20 16:45:40.624660 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:40.624643 2579 scope.go:117] "RemoveContainer" containerID="6c97ba341b51653d4187f5cb248a8dcd0c2c9e299a039cddfec2a10e9829815b" Jul 20 16:45:41.469911 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.469882 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:41.470341 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:41.470033 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:41.470341 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.470067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:41.470341 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:41.470136 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:41.629416 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.629384 2579 generic.go:356] "Generic (PLEG): container finished" podID="3fa635bd-8080-48a0-814d-9b794456bbe0" containerID="07a189b5127808046925a6cdb3320fe4c6e5643dd0d6d33ebe72ede5e66e5a25" exitCode=0 Jul 20 16:45:41.629576 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.629484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerDied","Data":"07a189b5127808046925a6cdb3320fe4c6e5643dd0d6d33ebe72ede5e66e5a25"} Jul 20 16:45:41.637448 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.637420 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:45:41.637884 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.637861 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" event={"ID":"15a867d1-5903-4d89-96da-868c00db9f62","Type":"ContainerStarted","Data":"5bb965756021b76cebac577692b9ad52ee4030d778478b372e1ec66624275ecb"} Jul 20 16:45:41.638093 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.638075 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:41.638164 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.638140 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:41.638222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.638186 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:41.655786 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.655760 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:41.655922 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.655871 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:45:41.683485 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.683389 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" podStartSLOduration=10.733718539 podStartE2EDuration="29.683374704s" totalImagesPullingTime="18.949656165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:41.683194787 +0000 UTC m=+30.697617064" watchObservedRunningTime="2026-07-20 16:45:41.683374704 +0000 UTC m=+30.697796960" Jul 20 16:45:41.976865 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.976795 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l2prr"] Jul 20 16:45:41.976994 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.976923 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:41.977042 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:41.977002 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:41.979594 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.979568 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lj79"] Jul 20 16:45:41.979712 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.979670 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:41.979783 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:41.979762 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:41.980182 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.980158 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r7zpb"] Jul 20 16:45:41.980262 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:41.980255 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:41.980392 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:41.980370 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:43.472298 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:43.472097 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:43.472794 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:43.472144 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:43.472794 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:43.472153 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:43.472794 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:43.472410 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:43.472794 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:43.472493 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:43.472794 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:43.472564 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:43.645140 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:43.645107 2579 generic.go:356] "Generic (PLEG): container finished" podID="3fa635bd-8080-48a0-814d-9b794456bbe0" containerID="147454b05537127fa672fe2c3b545bb3259598b1f5e3d30afaa004c825b11ef9" exitCode=0 Jul 20 16:45:43.645301 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:43.645186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerDied","Data":"147454b05537127fa672fe2c3b545bb3259598b1f5e3d30afaa004c825b11ef9"} Jul 20 16:45:45.195610 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:45.195578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:45.195610 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:45.195616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:45.196237 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.195733 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:45.196237 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.195733 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:45.196237 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.195794 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs podName:3e64ce67-d0d1-401a-9b1d-7d08a28e3585 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:17.19577748 +0000 UTC m=+66.210199741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs") pod "network-metrics-daemon-r7zpb" (UID: "3e64ce67-d0d1-401a-9b1d-7d08a28e3585") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 16:45:45.196237 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.195810 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret podName:416d3c60-3d18-4f9e-a38f-4e5b085467d9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:17.195803101 +0000 UTC m=+66.210225335 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret") pod "global-pull-secret-syncer-4lj79" (UID: "416d3c60-3d18-4f9e-a38f-4e5b085467d9") : object "kube-system"/"original-pull-secret" not registered Jul 20 16:45:45.296196 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:45.296156 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:45.296388 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.296267 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 16:45:45.296388 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.296291 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 16:45:45.296388 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.296323 2579 projected.go:196] Error preparing data for projected volume kube-api-access-52dcq for pod openshift-network-diagnostics/network-check-target-l2prr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:45.296507 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.296392 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq podName:ae6aa1ac-8909-4435-bde8-4253edaf7be6 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:17.296374078 +0000 UTC m=+66.310796314 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-52dcq" (UniqueName: "kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq") pod "network-check-target-l2prr" (UID: "ae6aa1ac-8909-4435-bde8-4253edaf7be6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 16:45:45.472480 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:45.472404 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:45.472630 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:45.472411 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:45.472630 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.472614 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7zpb" podUID="3e64ce67-d0d1-401a-9b1d-7d08a28e3585" Jul 20 16:45:45.472727 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.472506 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l2prr" podUID="ae6aa1ac-8909-4435-bde8-4253edaf7be6" Jul 20 16:45:45.472727 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:45.472412 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:45.472812 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:45.472725 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lj79" podUID="416d3c60-3d18-4f9e-a38f-4e5b085467d9" Jul 20 16:45:46.859795 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.859764 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-206.ec2.internal" event="NodeReady" Jul 20 16:45:46.860387 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.859927 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 16:45:46.894318 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.894282 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c8dcd5578-dzmn4"] Jul 20 16:45:46.919074 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.919042 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kc8wg"] Jul 20 16:45:46.919238 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.919152 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:46.922549 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.922520 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 16:45:46.922688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.922625 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 16:45:46.922776 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.922755 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2mkz5\"" Jul 20 16:45:46.922906 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.922862 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 16:45:46.929443 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.929422 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 16:45:46.934911 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.934892 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k7gcx"] Jul 20 16:45:46.935037 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.935022 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:46.938014 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.937995 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 16:45:46.938014 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.938014 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vsgmd\"" Jul 20 16:45:46.938173 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.938057 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 16:45:46.958347 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.958326 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c8dcd5578-dzmn4"] Jul 20 16:45:46.958454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.958364 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kc8wg"] Jul 20 16:45:46.958454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.958387 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k7gcx"] Jul 20 16:45:46.958454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.958415 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:46.962232 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.961546 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-47jm7\"" Jul 20 16:45:46.962232 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.961624 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 16:45:46.963793 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.963776 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 16:45:46.964061 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:46.964042 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 16:45:47.008163 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-installation-pull-secrets\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008363 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-ca-trust-extracted\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008363 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008363 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-bound-sa-token\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008506 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-trusted-ca\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008506 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7drv9\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-kube-api-access-7drv9\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008506 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008437 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-certificates\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.008506 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.008472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-image-registry-private-configuration\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109482 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-ca-trust-extracted\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109662 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109490 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109662 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-bound-sa-token\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109662 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.109637 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 16:45:47.109662 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.109662 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c8dcd5578-dzmn4: secret "image-registry-tls" not found Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109646 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f986628-3da0-4d06-853e-634332ca7563-config-volume\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.109724 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls podName:bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a nodeName:}" failed. No retries permitted until 2026-07-20 16:45:47.609703185 +0000 UTC m=+36.624125434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls") pod "image-registry-6c8dcd5578-dzmn4" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a") : secret "image-registry-tls" not found Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-trusted-ca\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109814 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7drv9\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-kube-api-access-7drv9\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109845 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j77jr\" (UniqueName: \"kubernetes.io/projected/61557d82-0ea0-426a-b5ee-8a24ad219788-kube-api-access-j77jr\") pod \"ingress-canary-k7gcx\" (UID: \"61557d82-0ea0-426a-b5ee-8a24ad219788\") " pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-certificates\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.109889 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61557d82-0ea0-426a-b5ee-8a24ad219788-cert\") pod \"ingress-canary-k7gcx\" (UID: \"61557d82-0ea0-426a-b5ee-8a24ad219788\") " pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.110180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.110180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.109916 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-ca-trust-extracted\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.110180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.110051 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-image-registry-private-configuration\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.110180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.110081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-installation-pull-secrets\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.110180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.110098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f986628-3da0-4d06-853e-634332ca7563-tmp-dir\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.110180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.110127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6nzf\" (UniqueName: \"kubernetes.io/projected/1f986628-3da0-4d06-853e-634332ca7563-kube-api-access-l6nzf\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.110445 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.110428 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-certificates\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.110783 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.110757 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-trusted-ca\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.114717 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.114691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-image-registry-private-configuration\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.114827 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.114702 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-installation-pull-secrets\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.119022 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.118994 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-bound-sa-token\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.120209 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.120187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7drv9\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-kube-api-access-7drv9\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.211033 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.210989 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6nzf\" (UniqueName: \"kubernetes.io/projected/1f986628-3da0-4d06-853e-634332ca7563-kube-api-access-l6nzf\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.211221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211056 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f986628-3da0-4d06-853e-634332ca7563-config-volume\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.211221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j77jr\" (UniqueName: \"kubernetes.io/projected/61557d82-0ea0-426a-b5ee-8a24ad219788-kube-api-access-j77jr\") pod \"ingress-canary-k7gcx\" (UID: \"61557d82-0ea0-426a-b5ee-8a24ad219788\") " pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.211221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211128 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61557d82-0ea0-426a-b5ee-8a24ad219788-cert\") pod \"ingress-canary-k7gcx\" (UID: \"61557d82-0ea0-426a-b5ee-8a24ad219788\") " pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.211221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211151 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.211221 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211199 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f986628-3da0-4d06-853e-634332ca7563-tmp-dir\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.211587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211570 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f986628-3da0-4d06-853e-634332ca7563-tmp-dir\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.211678 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.211663 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 16:45:47.211739 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.211726 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls podName:1f986628-3da0-4d06-853e-634332ca7563 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:47.711706606 +0000 UTC m=+36.726128842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls") pod "dns-default-kc8wg" (UID: "1f986628-3da0-4d06-853e-634332ca7563") : secret "dns-default-metrics-tls" not found Jul 20 16:45:47.211816 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.211793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f986628-3da0-4d06-853e-634332ca7563-config-volume\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.213975 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.213952 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61557d82-0ea0-426a-b5ee-8a24ad219788-cert\") pod \"ingress-canary-k7gcx\" (UID: \"61557d82-0ea0-426a-b5ee-8a24ad219788\") " pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.223729 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.223709 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6nzf\" (UniqueName: \"kubernetes.io/projected/1f986628-3da0-4d06-853e-634332ca7563-kube-api-access-l6nzf\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.223848 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.223767 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j77jr\" (UniqueName: \"kubernetes.io/projected/61557d82-0ea0-426a-b5ee-8a24ad219788-kube-api-access-j77jr\") pod \"ingress-canary-k7gcx\" (UID: \"61557d82-0ea0-426a-b5ee-8a24ad219788\") " pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.269079 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.269050 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k7gcx" Jul 20 16:45:47.469062 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.469028 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:45:47.469062 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.469028 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:45:47.469272 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.469028 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:45:47.472708 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.472682 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sqpl\"" Jul 20 16:45:47.472988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.472968 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vk4l\"" Jul 20 16:45:47.472988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.472976 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 16:45:47.473204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.473188 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 16:45:47.473254 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.473192 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 16:45:47.474088 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.474071 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 16:45:47.614492 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.614445 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:47.614698 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.614601 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 16:45:47.614698 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.614615 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c8dcd5578-dzmn4: secret "image-registry-tls" not found Jul 20 16:45:47.614698 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.614676 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls podName:bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a nodeName:}" failed. No retries permitted until 2026-07-20 16:45:48.614657204 +0000 UTC m=+37.629079439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls") pod "image-registry-6c8dcd5578-dzmn4" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a") : secret "image-registry-tls" not found Jul 20 16:45:47.715533 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:47.715495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:47.715727 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.715664 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 16:45:47.715798 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:47.715748 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls podName:1f986628-3da0-4d06-853e-634332ca7563 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:48.715728498 +0000 UTC m=+37.730150755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls") pod "dns-default-kc8wg" (UID: "1f986628-3da0-4d06-853e-634332ca7563") : secret "dns-default-metrics-tls" not found Jul 20 16:45:48.380454 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:48.379847 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k7gcx"] Jul 20 16:45:48.465236 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:48.465191 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61557d82_0ea0_426a_b5ee_8a24ad219788.slice/crio-8d6a041081ecb4835ab0aeafc74246a5fefea9248bc37df32a61133e14abbf95 WatchSource:0}: Error finding container 8d6a041081ecb4835ab0aeafc74246a5fefea9248bc37df32a61133e14abbf95: Status 404 returned error can't find the container with id 8d6a041081ecb4835ab0aeafc74246a5fefea9248bc37df32a61133e14abbf95 Jul 20 16:45:48.624227 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:48.624190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:48.624418 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:48.624391 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 16:45:48.624464 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:48.624424 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c8dcd5578-dzmn4: secret "image-registry-tls" not found Jul 20 16:45:48.624517 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:48.624505 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls podName:bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a nodeName:}" failed. No retries permitted until 2026-07-20 16:45:50.624478188 +0000 UTC m=+39.638900425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls") pod "image-registry-6c8dcd5578-dzmn4" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a") : secret "image-registry-tls" not found Jul 20 16:45:48.661270 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:48.661238 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerStarted","Data":"926511eb89c5524bd42fae1144bdfeb4de8baeb7773190a600a6753151422969"} Jul 20 16:45:48.662585 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:48.662560 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k7gcx" event={"ID":"61557d82-0ea0-426a-b5ee-8a24ad219788","Type":"ContainerStarted","Data":"8d6a041081ecb4835ab0aeafc74246a5fefea9248bc37df32a61133e14abbf95"} Jul 20 16:45:48.724664 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:48.724587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:48.724810 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:48.724759 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 16:45:48.724873 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:48.724839 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls podName:1f986628-3da0-4d06-853e-634332ca7563 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:50.724820138 +0000 UTC m=+39.739242378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls") pod "dns-default-kc8wg" (UID: "1f986628-3da0-4d06-853e-634332ca7563") : secret "dns-default-metrics-tls" not found Jul 20 16:45:49.397263 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.397227 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t"] Jul 20 16:45:49.405741 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.405712 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm"] Jul 20 16:45:49.405875 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.405840 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" Jul 20 16:45:49.409007 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.408988 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.409302 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.409288 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.409424 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.409409 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-v7m7j\"" Jul 20 16:45:49.414814 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.414791 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2"] Jul 20 16:45:49.414935 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.414916 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" Jul 20 16:45:49.417760 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.417612 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gp5wp\"" Jul 20 16:45:49.419845 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.419825 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c"] Jul 20 16:45:49.419990 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.419934 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.421694 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.421672 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6"] Jul 20 16:45:49.421820 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.421764 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:49.422793 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.422770 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 20 16:45:49.423000 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.422979 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 20 16:45:49.423533 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.423513 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-w46dh\"" Jul 20 16:45:49.423754 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.423737 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.423754 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.423749 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.425220 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425202 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t"] Jul 20 16:45:49.425340 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425230 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c"] Jul 20 16:45:49.425340 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425245 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm"] Jul 20 16:45:49.425340 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.425788 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425771 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sdxkt\"" Jul 20 16:45:49.425884 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425787 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.425884 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425820 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.425884 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.425853 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 20 16:45:49.428327 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.428261 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 20 16:45:49.428846 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.428504 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.428846 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.428550 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hxxrn\"" Jul 20 16:45:49.428846 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.428730 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 20 16:45:49.429045 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.429031 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.429402 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.429386 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6"] Jul 20 16:45:49.431730 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.431100 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2"] Jul 20 16:45:49.503639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.503605 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-dl6bh"] Jul 20 16:45:49.508524 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.508502 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt"] Jul 20 16:45:49.508661 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.508637 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.511881 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.511854 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 20 16:45:49.513647 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.513625 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.513785 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.513672 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 20 16:45:49.513869 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.513855 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.513972 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.513957 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nlx2l\"" Jul 20 16:45:49.517455 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.517439 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ztj6t"] Jul 20 16:45:49.517579 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.517564 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.520417 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.520397 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 20 16:45:49.520556 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.520416 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.520675 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.520661 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 20 16:45:49.521022 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.521004 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.522640 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.522623 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 20 16:45:49.522718 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.522640 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d4d6z\"" Jul 20 16:45:49.524003 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.523956 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d855b796b-mp8tb"] Jul 20 16:45:49.524104 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.524067 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.526916 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.526884 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 16:45:49.526916 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.526906 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-b5r8c\"" Jul 20 16:45:49.527355 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.527281 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 16:45:49.527615 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.527375 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.527727 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.527400 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.527941 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.527922 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-dl6bh"] Jul 20 16:45:49.528038 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.527960 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt"] Jul 20 16:45:49.528038 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.527990 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.529955 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.529935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.530043 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.529965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:49.530043 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.529984 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7469\" (UniqueName: \"kubernetes.io/projected/d2f842e7-7cb1-448d-8748-e36c249734dc-kube-api-access-t7469\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:49.530165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530113 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhr5k\" (UniqueName: \"kubernetes.io/projected/f09b4f5f-869c-422e-99e6-4b0df74d6733-kube-api-access-hhr5k\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.530165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530153 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl747\" (UniqueName: \"kubernetes.io/projected/755f8ae7-c289-4923-b1a7-e3ee1c09e7d7-kube-api-access-jl747\") pod \"network-check-source-7bc5b6bf85-rcfkm\" (UID: \"755f8ae7-c289-4923-b1a7-e3ee1c09e7d7\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" Jul 20 16:45:49.530261 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530199 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmhln\" (UniqueName: \"kubernetes.io/projected/19d59c2a-8667-4d18-a446-dc902d612c5f-kube-api-access-pmhln\") pod \"volume-data-source-validator-78cf4c6f6-xmj6t\" (UID: \"19d59c2a-8667-4d18-a446-dc902d612c5f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" Jul 20 16:45:49.530339 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530321 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f09b4f5f-869c-422e-99e6-4b0df74d6733-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.530398 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkkxp\" (UniqueName: \"kubernetes.io/projected/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-kube-api-access-fkkxp\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.530398 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530391 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f09b4f5f-869c-422e-99e6-4b0df74d6733-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.530468 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.530414 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-config\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.533037 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.533017 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 16:45:49.533183 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.533161 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 16:45:49.533607 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.533587 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 16:45:49.533709 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.533699 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 20 16:45:49.533837 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.533807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 20 16:45:49.534070 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.534034 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 20 16:45:49.534070 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.534055 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 20 16:45:49.534196 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.534153 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-84pch\"" Jul 20 16:45:49.535155 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.535029 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d855b796b-mp8tb"] Jul 20 16:45:49.542347 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.542328 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ztj6t"] Jul 20 16:45:49.631360 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631299 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:49.631552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-stats-auth\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.631552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631402 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl747\" (UniqueName: \"kubernetes.io/projected/755f8ae7-c289-4923-b1a7-e3ee1c09e7d7-kube-api-access-jl747\") pod \"network-check-source-7bc5b6bf85-rcfkm\" (UID: \"755f8ae7-c289-4923-b1a7-e3ee1c09e7d7\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" Jul 20 16:45:49.631552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhr5k\" (UniqueName: \"kubernetes.io/projected/f09b4f5f-869c-422e-99e6-4b0df74d6733-kube-api-access-hhr5k\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.631552 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.631449 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 16:45:49.631552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631494 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74d356e-e508-432c-a37c-e274dcd195b7-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.631552 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.631525 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls podName:d2f842e7-7cb1-448d-8748-e36c249734dc nodeName:}" failed. No retries permitted until 2026-07-20 16:45:50.131504911 +0000 UTC m=+39.145927157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-glp9c" (UID: "d2f842e7-7cb1-448d-8748-e36c249734dc") : secret "samples-operator-tls" not found Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cef8a584-a9c8-4df8-80e9-f61b935eb73d-serving-cert\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkkxp\" (UniqueName: \"kubernetes.io/projected/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-kube-api-access-fkkxp\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631623 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhd8n\" (UniqueName: \"kubernetes.io/projected/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-kube-api-access-hhd8n\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cef8a584-a9c8-4df8-80e9-f61b935eb73d-config\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f09b4f5f-869c-422e-99e6-4b0df74d6733-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.631849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cef8a584-a9c8-4df8-80e9-f61b935eb73d-trusted-ca\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631870 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631887 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-default-certificate\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631912 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74d356e-e508-432c-a37c-e274dcd195b7-service-ca-bundle\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7469\" (UniqueName: \"kubernetes.io/projected/d2f842e7-7cb1-448d-8748-e36c249734dc-kube-api-access-t7469\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.631948 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jwcf\" (UniqueName: \"kubernetes.io/projected/cef8a584-a9c8-4df8-80e9-f61b935eb73d-kube-api-access-4jwcf\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632002 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d356e-e508-432c-a37c-e274dcd195b7-tmp\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632055 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhln\" (UniqueName: \"kubernetes.io/projected/19d59c2a-8667-4d18-a446-dc902d612c5f-kube-api-access-pmhln\") pod \"volume-data-source-validator-78cf4c6f6-xmj6t\" (UID: \"19d59c2a-8667-4d18-a446-dc902d612c5f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh7zh\" (UniqueName: \"kubernetes.io/projected/e74d356e-e508-432c-a37c-e274dcd195b7-kube-api-access-kh7zh\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e74d356e-e508-432c-a37c-e274dcd195b7-snapshots\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.632166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f09b4f5f-869c-422e-99e6-4b0df74d6733-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.632618 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632219 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.632618 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632239 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f09b4f5f-869c-422e-99e6-4b0df74d6733-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.632704 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632633 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-config\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.632704 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632683 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e74d356e-e508-432c-a37c-e274dcd195b7-serving-cert\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.632791 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632716 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.632934 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vzh2\" (UniqueName: \"kubernetes.io/projected/22e6b30a-676c-4f8a-85e4-620986107327-kube-api-access-7vzh2\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.633001 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.632960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/22e6b30a-676c-4f8a-85e4-620986107327-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.633593 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.633336 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-config\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.637050 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.636744 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.639570 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.639551 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f09b4f5f-869c-422e-99e6-4b0df74d6733-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.644613 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.644588 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhr5k\" (UniqueName: \"kubernetes.io/projected/f09b4f5f-869c-422e-99e6-4b0df74d6733-kube-api-access-hhr5k\") pod \"kube-storage-version-migrator-operator-7658d9b749-4hmr6\" (UID: \"f09b4f5f-869c-422e-99e6-4b0df74d6733\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.644706 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.644638 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl747\" (UniqueName: \"kubernetes.io/projected/755f8ae7-c289-4923-b1a7-e3ee1c09e7d7-kube-api-access-jl747\") pod \"network-check-source-7bc5b6bf85-rcfkm\" (UID: \"755f8ae7-c289-4923-b1a7-e3ee1c09e7d7\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" Jul 20 16:45:49.644706 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.644653 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7469\" (UniqueName: \"kubernetes.io/projected/d2f842e7-7cb1-448d-8748-e36c249734dc-kube-api-access-t7469\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:49.644895 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.644872 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmhln\" (UniqueName: \"kubernetes.io/projected/19d59c2a-8667-4d18-a446-dc902d612c5f-kube-api-access-pmhln\") pod \"volume-data-source-validator-78cf4c6f6-xmj6t\" (UID: \"19d59c2a-8667-4d18-a446-dc902d612c5f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" Jul 20 16:45:49.644988 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.644967 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkkxp\" (UniqueName: \"kubernetes.io/projected/87d0d3ff-2190-44b9-8b8b-833c9a0865cf-kube-api-access-fkkxp\") pod \"service-ca-operator-5bcbc94c78-ljjp2\" (UID: \"87d0d3ff-2190-44b9-8b8b-833c9a0865cf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.667539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.667450 2579 generic.go:356] "Generic (PLEG): container finished" podID="3fa635bd-8080-48a0-814d-9b794456bbe0" containerID="926511eb89c5524bd42fae1144bdfeb4de8baeb7773190a600a6753151422969" exitCode=0 Jul 20 16:45:49.667539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.667522 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerDied","Data":"926511eb89c5524bd42fae1144bdfeb4de8baeb7773190a600a6753151422969"} Jul 20 16:45:49.720533 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.719985 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" Jul 20 16:45:49.727042 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.727016 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" Jul 20 16:45:49.733590 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.733705 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vzh2\" (UniqueName: \"kubernetes.io/projected/22e6b30a-676c-4f8a-85e4-620986107327-kube-api-access-7vzh2\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.733705 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/22e6b30a-676c-4f8a-85e4-620986107327-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.733705 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.733696 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:49.733850 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.733753 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls podName:22e6b30a-676c-4f8a-85e4-620986107327 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:50.233736996 +0000 UTC m=+39.248159230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-56dlt" (UID: "22e6b30a-676c-4f8a-85e4-620986107327") : secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:49.733850 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733818 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-stats-auth\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.733953 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733892 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74d356e-e508-432c-a37c-e274dcd195b7-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.733953 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733936 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cef8a584-a9c8-4df8-80e9-f61b935eb73d-serving-cert\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.734125 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.733978 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhd8n\" (UniqueName: \"kubernetes.io/projected/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-kube-api-access-hhd8n\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.734125 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cef8a584-a9c8-4df8-80e9-f61b935eb73d-config\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.734125 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.734125 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734076 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cef8a584-a9c8-4df8-80e9-f61b935eb73d-trusted-ca\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.734125 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734107 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-default-certificate\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74d356e-e508-432c-a37c-e274dcd195b7-service-ca-bundle\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jwcf\" (UniqueName: \"kubernetes.io/projected/cef8a584-a9c8-4df8-80e9-f61b935eb73d-kube-api-access-4jwcf\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d356e-e508-432c-a37c-e274dcd195b7-tmp\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734234 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh7zh\" (UniqueName: \"kubernetes.io/projected/e74d356e-e508-432c-a37c-e274dcd195b7-kube-api-access-kh7zh\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e74d356e-e508-432c-a37c-e274dcd195b7-snapshots\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734331 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e74d356e-e508-432c-a37c-e274dcd195b7-serving-cert\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.734391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734379 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/22e6b30a-676c-4f8a-85e4-620986107327-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.734808 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.734513 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:50.234497597 +0000 UTC m=+39.248919851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : configmap references non-existent config key: service-ca.crt Jul 20 16:45:49.734881 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734870 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74d356e-e508-432c-a37c-e274dcd195b7-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734944 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e74d356e-e508-432c-a37c-e274dcd195b7-tmp\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.734953 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e74d356e-e508-432c-a37c-e274dcd195b7-snapshots\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.735066 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cef8a584-a9c8-4df8-80e9-f61b935eb73d-config\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.735085 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.735334 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:49.735399 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:50.235385369 +0000 UTC m=+39.249807603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : secret "router-metrics-certs-default" not found Jul 20 16:45:49.735418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.735412 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cef8a584-a9c8-4df8-80e9-f61b935eb73d-trusted-ca\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.735803 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.735652 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74d356e-e508-432c-a37c-e274dcd195b7-service-ca-bundle\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.737918 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.737890 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e74d356e-e508-432c-a37c-e274dcd195b7-serving-cert\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.738115 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.738092 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-default-certificate\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.738504 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.738479 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cef8a584-a9c8-4df8-80e9-f61b935eb73d-serving-cert\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.738597 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.738584 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-stats-auth\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.744569 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.744543 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" Jul 20 16:45:49.746253 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.746227 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhd8n\" (UniqueName: \"kubernetes.io/projected/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-kube-api-access-hhd8n\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:49.746691 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.746672 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jwcf\" (UniqueName: \"kubernetes.io/projected/cef8a584-a9c8-4df8-80e9-f61b935eb73d-kube-api-access-4jwcf\") pod \"console-operator-565b7b569d-ztj6t\" (UID: \"cef8a584-a9c8-4df8-80e9-f61b935eb73d\") " pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.746796 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.746737 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vzh2\" (UniqueName: \"kubernetes.io/projected/22e6b30a-676c-4f8a-85e4-620986107327-kube-api-access-7vzh2\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:49.746863 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.746793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh7zh\" (UniqueName: \"kubernetes.io/projected/e74d356e-e508-432c-a37c-e274dcd195b7-kube-api-access-kh7zh\") pod \"insights-operator-555d9d7885-dl6bh\" (UID: \"e74d356e-e508-432c-a37c-e274dcd195b7\") " pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.820629 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.820146 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" Jul 20 16:45:49.844602 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.841950 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:49.927355 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.927327 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t"] Jul 20 16:45:49.933675 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.933654 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm"] Jul 20 16:45:49.949073 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.949047 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2"] Jul 20 16:45:49.969332 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:49.969284 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6"] Jul 20 16:45:49.974331 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:49.974280 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d0d3ff_2190_44b9_8b8b_833c9a0865cf.slice/crio-815fae7760e7d923824eac896940d867ab41f96c6f23beb4777c469bc1871c6f WatchSource:0}: Error finding container 815fae7760e7d923824eac896940d867ab41f96c6f23beb4777c469bc1871c6f: Status 404 returned error can't find the container with id 815fae7760e7d923824eac896940d867ab41f96c6f23beb4777c469bc1871c6f Jul 20 16:45:49.975572 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:49.975536 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d59c2a_8667_4d18_a446_dc902d612c5f.slice/crio-3cbe67fbb4cbf0c8f1a1197e0aabda0f92749b3557d7480071536ccf2b4425e9 WatchSource:0}: Error finding container 3cbe67fbb4cbf0c8f1a1197e0aabda0f92749b3557d7480071536ccf2b4425e9: Status 404 returned error can't find the container with id 3cbe67fbb4cbf0c8f1a1197e0aabda0f92749b3557d7480071536ccf2b4425e9 Jul 20 16:45:49.976693 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:49.976667 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755f8ae7_c289_4923_b1a7_e3ee1c09e7d7.slice/crio-2a4d406ffa1f086ce795299ec2e12b30a2068ce56125cb760b34308955b35986 WatchSource:0}: Error finding container 2a4d406ffa1f086ce795299ec2e12b30a2068ce56125cb760b34308955b35986: Status 404 returned error can't find the container with id 2a4d406ffa1f086ce795299ec2e12b30a2068ce56125cb760b34308955b35986 Jul 20 16:45:49.978296 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:49.978113 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09b4f5f_869c_422e_99e6_4b0df74d6733.slice/crio-0db5a4dcfa71181289d20c70a1db8fe5a0bc336238127574dce4fef8edce8381 WatchSource:0}: Error finding container 0db5a4dcfa71181289d20c70a1db8fe5a0bc336238127574dce4fef8edce8381: Status 404 returned error can't find the container with id 0db5a4dcfa71181289d20c70a1db8fe5a0bc336238127574dce4fef8edce8381 Jul 20 16:45:50.117015 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.116972 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ztj6t"] Jul 20 16:45:50.121370 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:50.121343 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef8a584_a9c8_4df8_80e9_f61b935eb73d.slice/crio-07c9681d17fc5a021fa107809e731b37856cfdf85e21f15fa5b89ba3d48e1a1a WatchSource:0}: Error finding container 07c9681d17fc5a021fa107809e731b37856cfdf85e21f15fa5b89ba3d48e1a1a: Status 404 returned error can't find the container with id 07c9681d17fc5a021fa107809e731b37856cfdf85e21f15fa5b89ba3d48e1a1a Jul 20 16:45:50.121982 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.121919 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-dl6bh"] Jul 20 16:45:50.124833 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:45:50.124812 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74d356e_e508_432c_a37c_e274dcd195b7.slice/crio-19178edeb3bd98ab4aaa0e9567d71b0dda23f89cefa1cb873421cd3e091346a2 WatchSource:0}: Error finding container 19178edeb3bd98ab4aaa0e9567d71b0dda23f89cefa1cb873421cd3e091346a2: Status 404 returned error can't find the container with id 19178edeb3bd98ab4aaa0e9567d71b0dda23f89cefa1cb873421cd3e091346a2 Jul 20 16:45:50.140363 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.140341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:50.140489 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.140468 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 16:45:50.140548 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.140522 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls podName:d2f842e7-7cb1-448d-8748-e36c249734dc nodeName:}" failed. No retries permitted until 2026-07-20 16:45:51.140503501 +0000 UTC m=+40.154925749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-glp9c" (UID: "d2f842e7-7cb1-448d-8748-e36c249734dc") : secret "samples-operator-tls" not found Jul 20 16:45:50.240931 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.240903 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:50.241058 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.240945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:50.241058 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.241030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:50.241188 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.241076 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:51.241058841 +0000 UTC m=+40.255481077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : configmap references non-existent config key: service-ca.crt Jul 20 16:45:50.241188 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.241087 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:50.241188 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.241128 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 16:45:50.241188 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.241143 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls podName:22e6b30a-676c-4f8a-85e4-620986107327 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:51.241131762 +0000 UTC m=+40.255554000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-56dlt" (UID: "22e6b30a-676c-4f8a-85e4-620986107327") : secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:50.241188 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.241172 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:51.241158318 +0000 UTC m=+40.255580554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : secret "router-metrics-certs-default" not found Jul 20 16:45:50.645536 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.645500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:50.646439 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.645659 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 16:45:50.646439 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.645678 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c8dcd5578-dzmn4: secret "image-registry-tls" not found Jul 20 16:45:50.646439 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.645740 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls podName:bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a nodeName:}" failed. No retries permitted until 2026-07-20 16:45:54.645719016 +0000 UTC m=+43.660141269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls") pod "image-registry-6c8dcd5578-dzmn4" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a") : secret "image-registry-tls" not found Jul 20 16:45:50.672423 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.672345 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" event={"ID":"cef8a584-a9c8-4df8-80e9-f61b935eb73d","Type":"ContainerStarted","Data":"07c9681d17fc5a021fa107809e731b37856cfdf85e21f15fa5b89ba3d48e1a1a"} Jul 20 16:45:50.673719 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.673519 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" event={"ID":"f09b4f5f-869c-422e-99e6-4b0df74d6733","Type":"ContainerStarted","Data":"0db5a4dcfa71181289d20c70a1db8fe5a0bc336238127574dce4fef8edce8381"} Jul 20 16:45:50.674969 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.674828 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" event={"ID":"e74d356e-e508-432c-a37c-e274dcd195b7","Type":"ContainerStarted","Data":"19178edeb3bd98ab4aaa0e9567d71b0dda23f89cefa1cb873421cd3e091346a2"} Jul 20 16:45:50.676541 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.676329 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" event={"ID":"87d0d3ff-2190-44b9-8b8b-833c9a0865cf","Type":"ContainerStarted","Data":"815fae7760e7d923824eac896940d867ab41f96c6f23beb4777c469bc1871c6f"} Jul 20 16:45:50.677697 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.677648 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" event={"ID":"19d59c2a-8667-4d18-a446-dc902d612c5f","Type":"ContainerStarted","Data":"3cbe67fbb4cbf0c8f1a1197e0aabda0f92749b3557d7480071536ccf2b4425e9"} Jul 20 16:45:50.681559 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.681528 2579 generic.go:356] "Generic (PLEG): container finished" podID="3fa635bd-8080-48a0-814d-9b794456bbe0" containerID="93768181d80f84ebbf7ae389bb623592bda4787c41699f8752c5720a4115219e" exitCode=0 Jul 20 16:45:50.681657 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.681614 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerDied","Data":"93768181d80f84ebbf7ae389bb623592bda4787c41699f8752c5720a4115219e"} Jul 20 16:45:50.685325 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.685279 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" event={"ID":"755f8ae7-c289-4923-b1a7-e3ee1c09e7d7","Type":"ContainerStarted","Data":"2a4d406ffa1f086ce795299ec2e12b30a2068ce56125cb760b34308955b35986"} Jul 20 16:45:50.747420 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:50.747136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:50.747940 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.747452 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 16:45:50.748050 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:50.747982 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls podName:1f986628-3da0-4d06-853e-634332ca7563 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:54.747961151 +0000 UTC m=+43.762383398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls") pod "dns-default-kc8wg" (UID: "1f986628-3da0-4d06-853e-634332ca7563") : secret "dns-default-metrics-tls" not found Jul 20 16:45:51.151754 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:51.151117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:51.151754 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.151345 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 16:45:51.151754 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.151414 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls podName:d2f842e7-7cb1-448d-8748-e36c249734dc nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.151394409 +0000 UTC m=+42.165816647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-glp9c" (UID: "d2f842e7-7cb1-448d-8748-e36c249734dc") : secret "samples-operator-tls" not found Jul 20 16:45:51.252050 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:51.252013 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:51.252202 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:51.252070 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:51.252202 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:51.252183 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:51.252338 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.252320 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 16:45:51.252420 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.252390 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.252369015 +0000 UTC m=+42.266791252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : secret "router-metrics-certs-default" not found Jul 20 16:45:51.252811 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.252789 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.25277324 +0000 UTC m=+42.267195481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : configmap references non-existent config key: service-ca.crt Jul 20 16:45:51.252881 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.252866 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:51.252933 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:51.252900 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls podName:22e6b30a-676c-4f8a-85e4-620986107327 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:53.252888419 +0000 UTC m=+42.267310659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-56dlt" (UID: "22e6b30a-676c-4f8a-85e4-620986107327") : secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:51.705961 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:51.705922 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" event={"ID":"3fa635bd-8080-48a0-814d-9b794456bbe0","Type":"ContainerStarted","Data":"96cbfb0b5e79868fa236c2443cee007c87cac0a359ce6e8c3238394877c23b63"} Jul 20 16:45:51.734158 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:51.734093 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9xmgj" podStartSLOduration=7.7664619550000005 podStartE2EDuration="39.734071296s" totalImagesPullingTime="31.967609341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:51.731724282 +0000 UTC m=+40.746146601" watchObservedRunningTime="2026-07-20 16:45:51.734071296 +0000 UTC m=+40.748493553" Jul 20 16:45:53.172829 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:53.172787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:53.173279 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.172966 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 16:45:53.173279 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.173055 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls podName:d2f842e7-7cb1-448d-8748-e36c249734dc nodeName:}" failed. No retries permitted until 2026-07-20 16:45:57.173033745 +0000 UTC m=+46.187455997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-glp9c" (UID: "d2f842e7-7cb1-448d-8748-e36c249734dc") : secret "samples-operator-tls" not found Jul 20 16:45:53.274043 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:53.273999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:53.274213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:53.274096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:53.274213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:53.274133 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:53.274213 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.274160 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 16:45:53.274338 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.274222 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:57.274204685 +0000 UTC m=+46.288626940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : secret "router-metrics-certs-default" not found Jul 20 16:45:53.274338 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.274256 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:53.274338 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.274267 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:45:57.274247704 +0000 UTC m=+46.288669960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : configmap references non-existent config key: service-ca.crt Jul 20 16:45:53.274448 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:53.274361 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls podName:22e6b30a-676c-4f8a-85e4-620986107327 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:57.274348525 +0000 UTC m=+46.288770759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-56dlt" (UID: "22e6b30a-676c-4f8a-85e4-620986107327") : secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:54.689375 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:54.689337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:45:54.689725 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:54.689486 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 16:45:54.689725 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:54.689500 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c8dcd5578-dzmn4: secret "image-registry-tls" not found Jul 20 16:45:54.689725 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:54.689556 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls podName:bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a nodeName:}" failed. No retries permitted until 2026-07-20 16:46:02.689539558 +0000 UTC m=+51.703961792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls") pod "image-registry-6c8dcd5578-dzmn4" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a") : secret "image-registry-tls" not found Jul 20 16:45:54.790504 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:54.790471 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:45:54.790643 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:54.790625 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 16:45:54.790703 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:54.790692 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls podName:1f986628-3da0-4d06-853e-634332ca7563 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:02.790672718 +0000 UTC m=+51.805094953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls") pod "dns-default-kc8wg" (UID: "1f986628-3da0-4d06-853e-634332ca7563") : secret "dns-default-metrics-tls" not found Jul 20 16:45:57.214141 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:57.214086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:45:57.214624 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.214272 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 16:45:57.214624 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.214366 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls podName:d2f842e7-7cb1-448d-8748-e36c249734dc nodeName:}" failed. No retries permitted until 2026-07-20 16:46:05.214347269 +0000 UTC m=+54.228769506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-glp9c" (UID: "d2f842e7-7cb1-448d-8748-e36c249734dc") : secret "samples-operator-tls" not found Jul 20 16:45:57.315393 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:57.315351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:45:57.315536 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:57.315459 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:57.315536 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:57.315507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:45:57.315536 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.315518 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:57.315645 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.315601 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls podName:22e6b30a-676c-4f8a-85e4-620986107327 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:05.315581578 +0000 UTC m=+54.330003835 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-56dlt" (UID: "22e6b30a-676c-4f8a-85e4-620986107327") : secret "cluster-monitoring-operator-tls" not found Jul 20 16:45:57.315645 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.315599 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 16:45:57.315645 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.315621 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:46:05.315611596 +0000 UTC m=+54.330033834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : configmap references non-existent config key: service-ca.crt Jul 20 16:45:57.315645 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:57.315641 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:46:05.315631991 +0000 UTC m=+54.330054226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : secret "router-metrics-certs-default" not found Jul 20 16:45:58.729595 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.729153 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k7gcx" event={"ID":"61557d82-0ea0-426a-b5ee-8a24ad219788","Type":"ContainerStarted","Data":"4e9f5f27dd9a8d24cd801f47e6a08cfe217e15c44e6805211e847e19aec71a95"} Jul 20 16:45:58.730987 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.730931 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" event={"ID":"755f8ae7-c289-4923-b1a7-e3ee1c09e7d7","Type":"ContainerStarted","Data":"994c6b57a334cb7cd9ddeb4106ae38086b8af2a0314b2a5a640c55d71cf479fc"} Jul 20 16:45:58.732779 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.732759 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/0.log" Jul 20 16:45:58.732902 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.732793 2579 generic.go:356] "Generic (PLEG): container finished" podID="cef8a584-a9c8-4df8-80e9-f61b935eb73d" containerID="b3b5a7f5153dd99cff97b6e56f309295f9bc7b6eeaf23cb382022f6b6a0e3be3" exitCode=255 Jul 20 16:45:58.733090 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.733075 2579 scope.go:117] "RemoveContainer" containerID="b3b5a7f5153dd99cff97b6e56f309295f9bc7b6eeaf23cb382022f6b6a0e3be3" Jul 20 16:45:58.733471 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.733279 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" event={"ID":"cef8a584-a9c8-4df8-80e9-f61b935eb73d","Type":"ContainerDied","Data":"b3b5a7f5153dd99cff97b6e56f309295f9bc7b6eeaf23cb382022f6b6a0e3be3"} Jul 20 16:45:58.734977 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.734955 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" event={"ID":"f09b4f5f-869c-422e-99e6-4b0df74d6733","Type":"ContainerStarted","Data":"e6fa07c328571a405ad0dd9c509853d6c44be05f704bcb5e745c6f6a4474a283"} Jul 20 16:45:58.737010 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.736962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" event={"ID":"e74d356e-e508-432c-a37c-e274dcd195b7","Type":"ContainerStarted","Data":"e672b5bd77391524b26f061c9bcd52df94331acca709bc6a9b5bc0324ff97d1b"} Jul 20 16:45:58.738630 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.738606 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" event={"ID":"87d0d3ff-2190-44b9-8b8b-833c9a0865cf","Type":"ContainerStarted","Data":"b92502f4f5636f3530734cc4241513be8b733e9508b2083a002593f17287bea1"} Jul 20 16:45:58.740165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.740143 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" event={"ID":"19d59c2a-8667-4d18-a446-dc902d612c5f","Type":"ContainerStarted","Data":"b7b452a3dca012b7aad1f0c046facccb996f0e692b564ead49e8a1b023f94923"} Jul 20 16:45:58.746192 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.746156 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k7gcx" podStartSLOduration=3.426965851 podStartE2EDuration="12.746134118s" totalImagesPullingTime="9.319168267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.745571447 +0000 UTC m=+47.759993724" watchObservedRunningTime="2026-07-20 16:45:58.746134118 +0000 UTC m=+47.760556374" Jul 20 16:45:58.767534 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.767478 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-rcfkm" podStartSLOduration=1.955835513 podStartE2EDuration="9.767453626s" totalImagesPullingTime="7.811618113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.766362472 +0000 UTC m=+47.780784730" watchObservedRunningTime="2026-07-20 16:45:58.767453626 +0000 UTC m=+47.781875883" Jul 20 16:45:58.783326 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.783210 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" podStartSLOduration=1.969618434 podStartE2EDuration="9.783188331s" totalImagesPullingTime="7.813569897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.781711594 +0000 UTC m=+47.796133852" watchObservedRunningTime="2026-07-20 16:45:58.783188331 +0000 UTC m=+47.797610588" Jul 20 16:45:58.823647 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.823597 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xmj6t" podStartSLOduration=2.013325313 podStartE2EDuration="9.823579471s" totalImagesPullingTime="7.810254158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.822371092 +0000 UTC m=+47.836793372" watchObservedRunningTime="2026-07-20 16:45:58.823579471 +0000 UTC m=+47.838001727" Jul 20 16:45:58.870755 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.869324 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" podStartSLOduration=2.058086437 podStartE2EDuration="9.869276174s" totalImagesPullingTime="7.811189737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.846430425 +0000 UTC m=+47.860852683" watchObservedRunningTime="2026-07-20 16:45:58.869276174 +0000 UTC m=+47.883698433" Jul 20 16:45:58.871355 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.871334 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xrd4w"] Jul 20 16:45:58.896690 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.887712 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" podStartSLOduration=2.223632536 podStartE2EDuration="9.887689097s" totalImagesPullingTime="7.664056561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:58.884721538 +0000 UTC m=+47.899143781" watchObservedRunningTime="2026-07-20 16:45:58.887689097 +0000 UTC m=+47.902111354" Jul 20 16:45:58.896690 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.896542 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xrd4w"] Jul 20 16:45:58.897058 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.897043 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:58.904460 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.904429 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tcv9s\"" Jul 20 16:45:58.904839 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.904452 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 16:45:58.905259 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.905087 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 16:45:58.931784 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.931749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-data-volume\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:58.931924 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.931797 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8j89\" (UniqueName: \"kubernetes.io/projected/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-api-access-z8j89\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:58.931924 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.931847 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:58.931924 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.931876 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-crio-socket\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:58.932107 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:58.932084 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033341 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033242 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-data-volume\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033341 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8j89\" (UniqueName: \"kubernetes.io/projected/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-api-access-z8j89\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033341 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033602 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-crio-socket\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033602 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033602 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.033505 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 16:45:59.033764 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.033607 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:59.533586917 +0000 UTC m=+48.548009173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : configmap "kube-rbac-proxy" not found Jul 20 16:45:59.033764 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033629 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-data-volume\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033764 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.033710 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-crio-socket\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.033764 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.033723 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 16:45:59.033973 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.033776 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:45:59.533761644 +0000 UTC m=+48.548183879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : secret "insights-runtime-extractor-tls" not found Jul 20 16:45:59.055344 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.055292 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8j89\" (UniqueName: \"kubernetes.io/projected/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-api-access-z8j89\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.537328 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.537277 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.537515 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.537383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:45:59.537515 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.537436 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 16:45:59.537515 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.537493 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 16:45:59.537515 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.537516 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:00.537499771 +0000 UTC m=+49.551922007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : secret "insights-runtime-extractor-tls" not found Jul 20 16:45:59.537668 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.537543 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:00.537531033 +0000 UTC m=+49.551953271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : configmap "kube-rbac-proxy" not found Jul 20 16:45:59.745353 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.745298 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:45:59.745766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.745750 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/0.log" Jul 20 16:45:59.745830 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.745788 2579 generic.go:356] "Generic (PLEG): container finished" podID="cef8a584-a9c8-4df8-80e9-f61b935eb73d" containerID="cdef3ec37003e63bcb3b953c9c8d5046bd1547e98f6b9f2abb705d11efe0c438" exitCode=255 Jul 20 16:45:59.745882 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.745867 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" event={"ID":"cef8a584-a9c8-4df8-80e9-f61b935eb73d","Type":"ContainerDied","Data":"cdef3ec37003e63bcb3b953c9c8d5046bd1547e98f6b9f2abb705d11efe0c438"} Jul 20 16:45:59.745945 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.745918 2579 scope.go:117] "RemoveContainer" containerID="b3b5a7f5153dd99cff97b6e56f309295f9bc7b6eeaf23cb382022f6b6a0e3be3" Jul 20 16:45:59.746133 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.746109 2579 scope.go:117] "RemoveContainer" containerID="cdef3ec37003e63bcb3b953c9c8d5046bd1547e98f6b9f2abb705d11efe0c438" Jul 20 16:45:59.746333 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:45:59.746273 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ztj6t_openshift-console-operator(cef8a584-a9c8-4df8-80e9-f61b935eb73d)\"" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" podUID="cef8a584-a9c8-4df8-80e9-f61b935eb73d" Jul 20 16:45:59.842655 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.842574 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:45:59.842655 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:45:59.842625 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:46:00.548542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:00.548507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:00.548752 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:00.548623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:00.548752 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:00.548661 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 16:46:00.548752 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:00.548718 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 16:46:00.548914 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:00.548757 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:02.548734766 +0000 UTC m=+51.563156999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : configmap "kube-rbac-proxy" not found Jul 20 16:46:00.548914 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:00.548776 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:02.548767643 +0000 UTC m=+51.563189877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : secret "insights-runtime-extractor-tls" not found Jul 20 16:46:00.663133 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:00.663105 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krvrr_a6a434ee-9394-44d3-bfa8-de9ee981cc63/dns-node-resolver/0.log" Jul 20 16:46:00.750196 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:00.750169 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:46:00.750595 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:00.750519 2579 scope.go:117] "RemoveContainer" containerID="cdef3ec37003e63bcb3b953c9c8d5046bd1547e98f6b9f2abb705d11efe0c438" Jul 20 16:46:00.750694 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:00.750675 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ztj6t_openshift-console-operator(cef8a584-a9c8-4df8-80e9-f61b935eb73d)\"" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" podUID="cef8a584-a9c8-4df8-80e9-f61b935eb73d" Jul 20 16:46:01.263069 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:01.263041 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-988qn_d06a057d-92d0-43c3-bd8c-487f8ab1e22b/node-ca/0.log" Jul 20 16:46:01.753274 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:01.753246 2579 scope.go:117] "RemoveContainer" containerID="cdef3ec37003e63bcb3b953c9c8d5046bd1547e98f6b9f2abb705d11efe0c438" Jul 20 16:46:01.753719 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:01.753462 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ztj6t_openshift-console-operator(cef8a584-a9c8-4df8-80e9-f61b935eb73d)\"" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" podUID="cef8a584-a9c8-4df8-80e9-f61b935eb73d" Jul 20 16:46:02.064356 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:02.064237 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k7gcx_61557d82-0ea0-426a-b5ee-8a24ad219788/serve-healthcheck-canary/0.log" Jul 20 16:46:02.568189 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:02.568147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:02.568395 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:02.568220 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:02.568395 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.568299 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 16:46:02.568395 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.568370 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 16:46:02.568395 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.568390 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:06.56837292 +0000 UTC m=+55.582795154 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : secret "insights-runtime-extractor-tls" not found Jul 20 16:46:02.568531 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.568471 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:06.568458229 +0000 UTC m=+55.582880465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : configmap "kube-rbac-proxy" not found Jul 20 16:46:02.769704 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:02.769673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:46:02.770122 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.769825 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 16:46:02.770122 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.769846 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c8dcd5578-dzmn4: secret "image-registry-tls" not found Jul 20 16:46:02.770122 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.769899 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls podName:bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a nodeName:}" failed. No retries permitted until 2026-07-20 16:46:18.769882829 +0000 UTC m=+67.784305065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls") pod "image-registry-6c8dcd5578-dzmn4" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a") : secret "image-registry-tls" not found Jul 20 16:46:02.870282 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:02.870196 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:46:02.870457 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.870376 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 16:46:02.870457 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:02.870445 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls podName:1f986628-3da0-4d06-853e-634332ca7563 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:18.870430341 +0000 UTC m=+67.884852574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls") pod "dns-default-kc8wg" (UID: "1f986628-3da0-4d06-853e-634332ca7563") : secret "dns-default-metrics-tls" not found Jul 20 16:46:05.291609 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:05.291568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:46:05.292062 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.291741 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 16:46:05.292062 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.291808 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls podName:d2f842e7-7cb1-448d-8748-e36c249734dc nodeName:}" failed. No retries permitted until 2026-07-20 16:46:21.291793939 +0000 UTC m=+70.306216172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-glp9c" (UID: "d2f842e7-7cb1-448d-8748-e36c249734dc") : secret "samples-operator-tls" not found Jul 20 16:46:05.392539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:05.392500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:05.392710 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:05.392581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:05.392710 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:05.392614 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:46:05.392710 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.392650 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 16:46:05.392710 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.392710 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:46:21.392695786 +0000 UTC m=+70.407118022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : secret "router-metrics-certs-default" not found Jul 20 16:46:05.392863 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.392729 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 16:46:05.392863 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.392740 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle podName:2d29f98c-cfe7-4be3-b9f7-3980bc6f718e nodeName:}" failed. No retries permitted until 2026-07-20 16:46:21.392722964 +0000 UTC m=+70.407145201 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle") pod "router-default-7d855b796b-mp8tb" (UID: "2d29f98c-cfe7-4be3-b9f7-3980bc6f718e") : configmap references non-existent config key: service-ca.crt Jul 20 16:46:05.392863 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:05.392770 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls podName:22e6b30a-676c-4f8a-85e4-620986107327 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:21.392759084 +0000 UTC m=+70.407181322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-56dlt" (UID: "22e6b30a-676c-4f8a-85e4-620986107327") : secret "cluster-monitoring-operator-tls" not found Jul 20 16:46:06.604355 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:06.604292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:06.604749 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:06.604395 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:06.604749 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:06.604453 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 16:46:06.604749 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:46:06.604557 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls podName:c3457581-df8a-483d-b5ef-7af9b1e5d4b9 nodeName:}" failed. No retries permitted until 2026-07-20 16:46:14.604536549 +0000 UTC m=+63.618958785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xrd4w" (UID: "c3457581-df8a-483d-b5ef-7af9b1e5d4b9") : secret "insights-runtime-extractor-tls" not found Jul 20 16:46:06.605042 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:06.605021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:13.656859 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:13.656824 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmxlb" Jul 20 16:46:14.672230 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:14.672197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:14.674786 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:14.674760 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c3457581-df8a-483d-b5ef-7af9b1e5d4b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xrd4w\" (UID: \"c3457581-df8a-483d-b5ef-7af9b1e5d4b9\") " pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:14.817818 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:14.817788 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tcv9s\"" Jul 20 16:46:14.826082 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:14.826063 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xrd4w" Jul 20 16:46:14.956334 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:14.956266 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xrd4w"] Jul 20 16:46:14.959072 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:14.959047 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3457581_df8a_483d_b5ef_7af9b1e5d4b9.slice/crio-f4a19418971d730a9201f86f89b585baa2282dbd36bec1c4f385b82e807e79a4 WatchSource:0}: Error finding container f4a19418971d730a9201f86f89b585baa2282dbd36bec1c4f385b82e807e79a4: Status 404 returned error can't find the container with id f4a19418971d730a9201f86f89b585baa2282dbd36bec1c4f385b82e807e79a4 Jul 20 16:46:15.798132 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:15.798006 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xrd4w" event={"ID":"c3457581-df8a-483d-b5ef-7af9b1e5d4b9","Type":"ContainerStarted","Data":"f4a19418971d730a9201f86f89b585baa2282dbd36bec1c4f385b82e807e79a4"} Jul 20 16:46:17.203785 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.203739 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:46:17.204332 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.203800 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:46:17.206776 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.206749 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 16:46:17.206934 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.206847 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 16:46:17.216995 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.216964 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/416d3c60-3d18-4f9e-a38f-4e5b085467d9-original-pull-secret\") pod \"global-pull-secret-syncer-4lj79\" (UID: \"416d3c60-3d18-4f9e-a38f-4e5b085467d9\") " pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:46:17.217185 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.217165 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e64ce67-d0d1-401a-9b1d-7d08a28e3585-metrics-certs\") pod \"network-metrics-daemon-r7zpb\" (UID: \"3e64ce67-d0d1-401a-9b1d-7d08a28e3585\") " pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:46:17.305250 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.305209 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:46:17.308152 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.308121 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52dcq\" (UniqueName: \"kubernetes.io/projected/ae6aa1ac-8909-4435-bde8-4253edaf7be6-kube-api-access-52dcq\") pod \"network-check-target-l2prr\" (UID: \"ae6aa1ac-8909-4435-bde8-4253edaf7be6\") " pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:46:17.469742 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.469664 2579 scope.go:117] "RemoveContainer" containerID="cdef3ec37003e63bcb3b953c9c8d5046bd1547e98f6b9f2abb705d11efe0c438" Jul 20 16:46:17.482002 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.481974 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lj79" Jul 20 16:46:17.493030 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.493005 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sqpl\"" Jul 20 16:46:17.499215 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.499191 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9vk4l\"" Jul 20 16:46:17.500217 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.500190 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7zpb" Jul 20 16:46:17.507285 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:17.507262 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:46:18.821330 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:18.821279 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:46:18.824082 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:18.824058 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"image-registry-6c8dcd5578-dzmn4\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:46:18.922411 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:18.922375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:46:18.924987 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:18.924954 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f986628-3da0-4d06-853e-634332ca7563-metrics-tls\") pod \"dns-default-kc8wg\" (UID: \"1f986628-3da0-4d06-853e-634332ca7563\") " pod="openshift-dns/dns-default-kc8wg" Jul 20 16:46:19.035521 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:19.035485 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2mkz5\"" Jul 20 16:46:19.043401 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:19.043375 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:46:19.047470 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:19.047451 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vsgmd\"" Jul 20 16:46:19.054811 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:19.054794 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kc8wg" Jul 20 16:46:20.351779 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.351699 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kc8wg"] Jul 20 16:46:20.357170 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:20.357094 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f986628_3da0_4d06_853e_634332ca7563.slice/crio-aac786fc0a3d9c04c4ebbd0f2789c7f6a97dced8f1a5098d8fbe0b8599c3bfeb WatchSource:0}: Error finding container aac786fc0a3d9c04c4ebbd0f2789c7f6a97dced8f1a5098d8fbe0b8599c3bfeb: Status 404 returned error can't find the container with id aac786fc0a3d9c04c4ebbd0f2789c7f6a97dced8f1a5098d8fbe0b8599c3bfeb Jul 20 16:46:20.373840 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.373759 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c8dcd5578-dzmn4"] Jul 20 16:46:20.377634 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:20.377590 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc272cd8_a7a5_4bad_aa0e_243b5c6cf99a.slice/crio-a9d47434c43a26a7e18b4dead9a6bf6a0c365313ffb18e5f444de916a3b76a81 WatchSource:0}: Error finding container a9d47434c43a26a7e18b4dead9a6bf6a0c365313ffb18e5f444de916a3b76a81: Status 404 returned error can't find the container with id a9d47434c43a26a7e18b4dead9a6bf6a0c365313ffb18e5f444de916a3b76a81 Jul 20 16:46:20.391100 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.391074 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lj79"] Jul 20 16:46:20.394408 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:20.394300 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416d3c60_3d18_4f9e_a38f_4e5b085467d9.slice/crio-d35d725a340211fd05f3ae595979c3602db6eee8c2563be66115708bbf2b6f71 WatchSource:0}: Error finding container d35d725a340211fd05f3ae595979c3602db6eee8c2563be66115708bbf2b6f71: Status 404 returned error can't find the container with id d35d725a340211fd05f3ae595979c3602db6eee8c2563be66115708bbf2b6f71 Jul 20 16:46:20.422408 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.422378 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l2prr"] Jul 20 16:46:20.432392 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:20.432366 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6aa1ac_8909_4435_bde8_4253edaf7be6.slice/crio-157f05308e9e9b25f04d37122e54e38e25bd64eab082db33e58c45ce972c9292 WatchSource:0}: Error finding container 157f05308e9e9b25f04d37122e54e38e25bd64eab082db33e58c45ce972c9292: Status 404 returned error can't find the container with id 157f05308e9e9b25f04d37122e54e38e25bd64eab082db33e58c45ce972c9292 Jul 20 16:46:20.590725 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.590687 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r7zpb"] Jul 20 16:46:20.594101 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:20.594074 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e64ce67_d0d1_401a_9b1d_7d08a28e3585.slice/crio-e975469950d368f4aeab87d9d482cbd235b7a837006272f3493a93244819e06e WatchSource:0}: Error finding container e975469950d368f4aeab87d9d482cbd235b7a837006272f3493a93244819e06e: Status 404 returned error can't find the container with id e975469950d368f4aeab87d9d482cbd235b7a837006272f3493a93244819e06e Jul 20 16:46:20.818921 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.818882 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xrd4w" event={"ID":"c3457581-df8a-483d-b5ef-7af9b1e5d4b9","Type":"ContainerStarted","Data":"04a63d6542866516e19bee11866d58da30387a16a9ec37f658bd1c5f18c30d35"} Jul 20 16:46:20.821102 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.821072 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc8wg" event={"ID":"1f986628-3da0-4d06-853e-634332ca7563","Type":"ContainerStarted","Data":"aac786fc0a3d9c04c4ebbd0f2789c7f6a97dced8f1a5098d8fbe0b8599c3bfeb"} Jul 20 16:46:20.824448 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.824348 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:46:20.824448 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.824434 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" event={"ID":"cef8a584-a9c8-4df8-80e9-f61b935eb73d","Type":"ContainerStarted","Data":"cd656dd8aabc86da7da190e12cf6edc311b278bcd5bfcc3ee36395b0d664e424"} Jul 20 16:46:20.825193 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.824944 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:46:20.828097 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.828070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l2prr" event={"ID":"ae6aa1ac-8909-4435-bde8-4253edaf7be6","Type":"ContainerStarted","Data":"4881f5aa67cb3b56175a9eb8baea504251a6fafde0c62ec76ee5ef61915f17f6"} Jul 20 16:46:20.828213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.828116 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l2prr" event={"ID":"ae6aa1ac-8909-4435-bde8-4253edaf7be6","Type":"ContainerStarted","Data":"157f05308e9e9b25f04d37122e54e38e25bd64eab082db33e58c45ce972c9292"} Jul 20 16:46:20.828213 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.828160 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:46:20.830982 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.830955 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7zpb" event={"ID":"3e64ce67-d0d1-401a-9b1d-7d08a28e3585","Type":"ContainerStarted","Data":"e975469950d368f4aeab87d9d482cbd235b7a837006272f3493a93244819e06e"} Jul 20 16:46:20.833124 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.832752 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" event={"ID":"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a","Type":"ContainerStarted","Data":"6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6"} Jul 20 16:46:20.833124 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.832807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" event={"ID":"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a","Type":"ContainerStarted","Data":"a9d47434c43a26a7e18b4dead9a6bf6a0c365313ffb18e5f444de916a3b76a81"} Jul 20 16:46:20.833283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.833268 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:46:20.834888 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.834831 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lj79" event={"ID":"416d3c60-3d18-4f9e-a38f-4e5b085467d9","Type":"ContainerStarted","Data":"d35d725a340211fd05f3ae595979c3602db6eee8c2563be66115708bbf2b6f71"} Jul 20 16:46:20.853651 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.853590 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" podStartSLOduration=24.17969674 podStartE2EDuration="31.853569142s" totalImagesPullingTime="7.673872402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:20.845051589 +0000 UTC m=+69.859473846" watchObservedRunningTime="2026-07-20 16:46:20.853569142 +0000 UTC m=+69.867991398" Jul 20 16:46:20.867235 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.866643 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l2prr" podStartSLOduration=68.866620875 podStartE2EDuration="1m8.866620875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:20.865625725 +0000 UTC m=+69.880047983" watchObservedRunningTime="2026-07-20 16:46:20.866620875 +0000 UTC m=+69.881043133" Jul 20 16:46:20.888364 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:20.887898 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" podStartSLOduration=68.887876853 podStartE2EDuration="1m8.887876853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:20.885765121 +0000 UTC m=+69.900187376" watchObservedRunningTime="2026-07-20 16:46:20.887876853 +0000 UTC m=+69.902299093" Jul 20 16:46:21.347073 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.347029 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:46:21.352945 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.352887 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2f842e7-7cb1-448d-8748-e36c249734dc-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-glp9c\" (UID: \"d2f842e7-7cb1-448d-8748-e36c249734dc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:46:21.447750 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.447707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:21.447937 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.447777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:46:21.447937 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.447847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:21.449011 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.448924 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-service-ca-bundle\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:21.464821 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.451285 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/22e6b30a-676c-4f8a-85e4-620986107327-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-56dlt\" (UID: \"22e6b30a-676c-4f8a-85e4-620986107327\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:46:21.467660 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.465496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d29f98c-cfe7-4be3-b9f7-3980bc6f718e-metrics-certs\") pod \"router-default-7d855b796b-mp8tb\" (UID: \"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e\") " pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:21.545132 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.545100 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sdxkt\"" Jul 20 16:46:21.552844 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.552815 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" Jul 20 16:46:21.639560 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.633689 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d4d6z\"" Jul 20 16:46:21.641473 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.641429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" Jul 20 16:46:21.648058 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.647884 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-84pch\"" Jul 20 16:46:21.657344 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.656895 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:21.764599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.763467 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c"] Jul 20 16:46:21.825993 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.825608 2579 patch_prober.go:28] interesting pod/console-operator-565b7b569d-ztj6t container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.17:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 16:46:21.825993 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.825675 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" podUID="cef8a584-a9c8-4df8-80e9-f61b935eb73d" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.17:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 20 16:46:21.883176 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.883108 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt"] Jul 20 16:46:21.914680 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:21.914398 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-ztj6t" Jul 20 16:46:22.113982 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.113945 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-xjm4c"] Jul 20 16:46:22.117778 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.116755 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:22.120283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.120220 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 16:46:22.120893 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.120686 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 16:46:22.120893 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.120686 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qt22z\"" Jul 20 16:46:22.125838 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.125647 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-xjm4c"] Jul 20 16:46:22.256617 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.256593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdz4q\" (UniqueName: \"kubernetes.io/projected/b0203aa3-6bb1-44ea-9b29-cfd518891282-kube-api-access-jdz4q\") pod \"downloads-5bc67dcd46-xjm4c\" (UID: \"b0203aa3-6bb1-44ea-9b29-cfd518891282\") " pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:22.357854 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.357261 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdz4q\" (UniqueName: \"kubernetes.io/projected/b0203aa3-6bb1-44ea-9b29-cfd518891282-kube-api-access-jdz4q\") pod \"downloads-5bc67dcd46-xjm4c\" (UID: \"b0203aa3-6bb1-44ea-9b29-cfd518891282\") " pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:22.367281 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.367226 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdz4q\" (UniqueName: \"kubernetes.io/projected/b0203aa3-6bb1-44ea-9b29-cfd518891282-kube-api-access-jdz4q\") pod \"downloads-5bc67dcd46-xjm4c\" (UID: \"b0203aa3-6bb1-44ea-9b29-cfd518891282\") " pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:22.376244 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.376205 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d855b796b-mp8tb"] Jul 20 16:46:22.438177 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.437070 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:22.633718 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.633619 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-xjm4c"] Jul 20 16:46:22.853024 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.850646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" event={"ID":"22e6b30a-676c-4f8a-85e4-620986107327","Type":"ContainerStarted","Data":"b416bf325e28ab589bad050d14255793b740b136c94d31326dc5b216186eaccc"} Jul 20 16:46:22.854898 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.854863 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7zpb" event={"ID":"3e64ce67-d0d1-401a-9b1d-7d08a28e3585","Type":"ContainerStarted","Data":"0845490f7fbcc0b5754d420ba64b05911ee19d05adb327c4241293cc51899d10"} Jul 20 16:46:22.855047 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.854928 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7zpb" event={"ID":"3e64ce67-d0d1-401a-9b1d-7d08a28e3585","Type":"ContainerStarted","Data":"e8b4545dcea5e5070dce07fb26a35a356391c2cc9ffa7eb35e0513a52ef9d72f"} Jul 20 16:46:22.857459 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.857431 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d855b796b-mp8tb" event={"ID":"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e","Type":"ContainerStarted","Data":"5e40a0f0e6c3da9b43080afeb16f9ada79902904874a9d452181f1b5d169ff71"} Jul 20 16:46:22.857588 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.857484 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d855b796b-mp8tb" event={"ID":"2d29f98c-cfe7-4be3-b9f7-3980bc6f718e","Type":"ContainerStarted","Data":"ee334855b496fd9c0bc144ee9fd3525593e705dfb6dab6e951c5d2c688e5ff36"} Jul 20 16:46:22.858803 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.858776 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" event={"ID":"d2f842e7-7cb1-448d-8748-e36c249734dc","Type":"ContainerStarted","Data":"5016464ab8a4bcd907cf998bd7d89e14ba6d635cc58fb816d6b5c60f5b33276c"} Jul 20 16:46:22.875174 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.875110 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r7zpb" podStartSLOduration=69.222381857 podStartE2EDuration="1m10.875091209s" totalImagesPullingTime="1.652709352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:22.874053605 +0000 UTC m=+71.888475885" watchObservedRunningTime="2026-07-20 16:46:22.875091209 +0000 UTC m=+71.889513467" Jul 20 16:46:22.904396 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:22.904300 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d855b796b-mp8tb" podStartSLOduration=33.904279698 podStartE2EDuration="33.904279698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:22.903991653 +0000 UTC m=+71.918413934" watchObservedRunningTime="2026-07-20 16:46:22.904279698 +0000 UTC m=+71.918701951" Jul 20 16:46:23.657767 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:23.657526 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:23.661882 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:23.661676 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:23.865414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:23.865299 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:23.867042 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:23.866819 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d855b796b-mp8tb" Jul 20 16:46:24.629463 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:24.629430 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0203aa3_6bb1_44ea_9b29_cfd518891282.slice/crio-0bb10eb74b8ab94ec7ccc4de817f300a18f17154c61020bf098c9d039865d472 WatchSource:0}: Error finding container 0bb10eb74b8ab94ec7ccc4de817f300a18f17154c61020bf098c9d039865d472: Status 404 returned error can't find the container with id 0bb10eb74b8ab94ec7ccc4de817f300a18f17154c61020bf098c9d039865d472 Jul 20 16:46:24.868449 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:24.868411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-xjm4c" event={"ID":"b0203aa3-6bb1-44ea-9b29-cfd518891282","Type":"ContainerStarted","Data":"0bb10eb74b8ab94ec7ccc4de817f300a18f17154c61020bf098c9d039865d472"} Jul 20 16:46:27.883184 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:27.883136 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc8wg" event={"ID":"1f986628-3da0-4d06-853e-634332ca7563","Type":"ContainerStarted","Data":"403ce508690870276aea86ad2a20d8e2df0f89aea874801befc02749e2014a5d"} Jul 20 16:46:27.884824 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:27.884794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lj79" event={"ID":"416d3c60-3d18-4f9e-a38f-4e5b085467d9","Type":"ContainerStarted","Data":"de5df7769d8566779fc18d263a1d051e4759a1e6701b0c548469afc2061118f3"} Jul 20 16:46:27.900901 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:27.900855 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4lj79" podStartSLOduration=68.991135321 podStartE2EDuration="1m15.900839282s" totalImagesPullingTime="6.909703961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:27.899947815 +0000 UTC m=+76.914370084" watchObservedRunningTime="2026-07-20 16:46:27.900839282 +0000 UTC m=+76.915261537" Jul 20 16:46:29.896720 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.896600 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kc8wg" event={"ID":"1f986628-3da0-4d06-853e-634332ca7563","Type":"ContainerStarted","Data":"18a0382969e17918f0bec7c5a1a59e5e42228b768d39ae81d652bb9160abb9d7"} Jul 20 16:46:29.896720 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.896668 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kc8wg" Jul 20 16:46:29.899246 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.899212 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" event={"ID":"22e6b30a-676c-4f8a-85e4-620986107327","Type":"ContainerStarted","Data":"fe4fe9286aa817a7059e5028790ccdea9d2546a2f273f95dd0fb1f828871c5a1"} Jul 20 16:46:29.902432 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.902406 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" event={"ID":"d2f842e7-7cb1-448d-8748-e36c249734dc","Type":"ContainerStarted","Data":"1b4ca7d802b0b41784ee52e7da37322b621f8289bca7a568d8287e560e83efec"} Jul 20 16:46:29.902539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.902459 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" event={"ID":"d2f842e7-7cb1-448d-8748-e36c249734dc","Type":"ContainerStarted","Data":"06779fa6cc9e7c3a9e222976b1f0ce014d2d9bc6a0f2124fde0c28a411223e46"} Jul 20 16:46:29.904452 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.904430 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xrd4w" event={"ID":"c3457581-df8a-483d-b5ef-7af9b1e5d4b9","Type":"ContainerStarted","Data":"f634b8314ec4cfd0bf17321f00d8ee1b8941bffa92e2c4b7d8ade6ee179a6cd0"} Jul 20 16:46:29.915498 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.915448 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kc8wg" podStartSLOduration=36.983164739 podStartE2EDuration="43.915431347s" totalImagesPullingTime="6.932266608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:29.913675952 +0000 UTC m=+78.928098208" watchObservedRunningTime="2026-07-20 16:46:29.915431347 +0000 UTC m=+78.929853602" Jul 20 16:46:29.930725 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.930679 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-glp9c" podStartSLOduration=34.006015612 podStartE2EDuration="40.930658135s" totalImagesPullingTime="6.924642523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:29.92954819 +0000 UTC m=+78.943970447" watchObservedRunningTime="2026-07-20 16:46:29.930658135 +0000 UTC m=+78.945080393" Jul 20 16:46:29.955618 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:29.955559 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-56dlt" podStartSLOduration=33.950287597 podStartE2EDuration="40.955536973s" totalImagesPullingTime="7.005249376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:29.955169742 +0000 UTC m=+78.969592022" watchObservedRunningTime="2026-07-20 16:46:29.955536973 +0000 UTC m=+78.969959230" Jul 20 16:46:33.924114 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:33.924071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xrd4w" event={"ID":"c3457581-df8a-483d-b5ef-7af9b1e5d4b9","Type":"ContainerStarted","Data":"d083793f2e4275689a11e2ab1b8f63b17d0f8e1e578d7b9a39e18eb9a5fdf4b0"} Jul 20 16:46:33.943302 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:33.943252 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xrd4w" podStartSLOduration=17.612187185 podStartE2EDuration="35.943235428s" totalImagesPullingTime="18.331048243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:45:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:33.942441301 +0000 UTC m=+82.956863582" watchObservedRunningTime="2026-07-20 16:46:33.943235428 +0000 UTC m=+82.957657684" Jul 20 16:46:39.912506 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:39.912465 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kc8wg" Jul 20 16:46:41.848127 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:41.848096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:46:44.971404 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:44.971354 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-xjm4c" event={"ID":"b0203aa3-6bb1-44ea-9b29-cfd518891282","Type":"ContainerStarted","Data":"d199b946d34af8d7bbffe9f53ad318b9ed9d60017dae2ac2ed3b173b89215ee2"} Jul 20 16:46:44.972302 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:44.971573 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:44.984772 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:44.984734 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-xjm4c" Jul 20 16:46:44.992208 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:44.992145 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-xjm4c" podStartSLOduration=3.182545705 podStartE2EDuration="22.992123598s" totalImagesPullingTime="19.809577893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:44.99053727 +0000 UTC m=+94.004959527" watchObservedRunningTime="2026-07-20 16:46:44.992123598 +0000 UTC m=+94.006545855" Jul 20 16:46:45.288908 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.288825 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8zc4l"] Jul 20 16:46:45.323410 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.323376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.326503 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.326459 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 16:46:45.326949 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.326751 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 16:46:45.326949 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.326858 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 16:46:45.327161 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.327142 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2tw6m\"" Jul 20 16:46:45.328110 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.328088 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.346835 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5zjj\" (UniqueName: \"kubernetes.io/projected/5cb6683c-72ce-47b7-a905-552a3d6133d3-kube-api-access-j5zjj\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.346875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-textfile\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.346915 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-root\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.346949 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-sys\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.346976 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-wtmp\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.347000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-tls\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.347048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347625 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.347094 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.347625 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.347127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb6683c-72ce-47b7-a905-552a3d6133d3-metrics-client-ca\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.368504 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.368467 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c8dcd5578-dzmn4"] Jul 20 16:46:45.447979 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.447925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5zjj\" (UniqueName: \"kubernetes.io/projected/5cb6683c-72ce-47b7-a905-552a3d6133d3-kube-api-access-j5zjj\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.447979 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.447976 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-textfile\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-root\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448044 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-sys\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448064 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-wtmp\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448081 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-tls\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448127 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448162 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448222 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb6683c-72ce-47b7-a905-552a3d6133d3-metrics-client-ca\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-sys\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-root\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448876 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448784 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-wtmp\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448876 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448829 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-textfile\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.448942 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.448899 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb6683c-72ce-47b7-a905-552a3d6133d3-metrics-client-ca\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.449450 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.449398 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.451703 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.451679 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-tls\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.451783 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.451711 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb6683c-72ce-47b7-a905-552a3d6133d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.483917 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.483880 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5zjj\" (UniqueName: \"kubernetes.io/projected/5cb6683c-72ce-47b7-a905-552a3d6133d3-kube-api-access-j5zjj\") pod \"node-exporter-8zc4l\" (UID: \"5cb6683c-72ce-47b7-a905-552a3d6133d3\") " pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.636087 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.635844 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8zc4l" Jul 20 16:46:45.672541 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:45.672504 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb6683c_72ce_47b7_a905_552a3d6133d3.slice/crio-91bdc40d9f490ba1a294764377324c842d1f28a26fcd8f82ec8b7cd4e576b1a5 WatchSource:0}: Error finding container 91bdc40d9f490ba1a294764377324c842d1f28a26fcd8f82ec8b7cd4e576b1a5: Status 404 returned error can't find the container with id 91bdc40d9f490ba1a294764377324c842d1f28a26fcd8f82ec8b7cd4e576b1a5 Jul 20 16:46:45.978483 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:45.978358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8zc4l" event={"ID":"5cb6683c-72ce-47b7-a905-552a3d6133d3","Type":"ContainerStarted","Data":"91bdc40d9f490ba1a294764377324c842d1f28a26fcd8f82ec8b7cd4e576b1a5"} Jul 20 16:46:47.989413 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:47.989285 2579 generic.go:356] "Generic (PLEG): container finished" podID="5cb6683c-72ce-47b7-a905-552a3d6133d3" containerID="a7a27438ca0e0ac3644ada39cf1115c87b72de3f899c4cd420458f57f5b3972b" exitCode=0 Jul 20 16:46:47.989413 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:47.989381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8zc4l" event={"ID":"5cb6683c-72ce-47b7-a905-552a3d6133d3","Type":"ContainerDied","Data":"a7a27438ca0e0ac3644ada39cf1115c87b72de3f899c4cd420458f57f5b3972b"} Jul 20 16:46:48.997418 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:48.997379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8zc4l" event={"ID":"5cb6683c-72ce-47b7-a905-552a3d6133d3","Type":"ContainerStarted","Data":"2927abbcb5e312673fd8c30a1a08c043067012cbf8f14515a39680347e66a44d"} Jul 20 16:46:48.997934 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:48.997453 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8zc4l" event={"ID":"5cb6683c-72ce-47b7-a905-552a3d6133d3","Type":"ContainerStarted","Data":"e409a1dd7cafe1dcaee8c66c9c5f498a8b0d8212b7483dedfdc771fb5903ddd8"} Jul 20 16:46:49.022562 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:49.022492 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8zc4l" podStartSLOduration=2.124009924 podStartE2EDuration="4.022467978s" totalImagesPullingTime="1.791934808s" totalInitContainerRuntime="106.523246ms" isStatefulPod=true podCreationTimestamp="2026-07-20 16:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:46:49.022081597 +0000 UTC m=+98.036503904" watchObservedRunningTime="2026-07-20 16:46:49.022467978 +0000 UTC m=+98.036890236" Jul 20 16:46:51.501180 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.498608 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:46:51.546600 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.546557 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:46:51.546792 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.546674 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.550167 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.550134 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 20 16:46:51.550340 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.550168 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 20 16:46:51.550654 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.550631 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 20 16:46:51.551036 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551015 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h6kkk\"" Jul 20 16:46:51.551174 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551019 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 20 16:46:51.551248 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 20 16:46:51.551372 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551358 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 20 16:46:51.551441 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551159 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 20 16:46:51.551441 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551393 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 20 16:46:51.551543 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551519 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 20 16:46:51.551844 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551685 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e5j2kiuj2hmto\"" Jul 20 16:46:51.551844 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.551791 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 20 16:46:51.552566 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.552545 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 20 16:46:51.552702 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.552582 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 20 16:46:51.556821 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.556798 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 20 16:46:51.557234 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.557214 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 16:46:51.558591 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.558569 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 20 16:46:51.607297 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607502 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607334 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607502 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607366 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsfhk\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-kube-api-access-rsfhk\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607502 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607502 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607502 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607422 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607560 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607590 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607655 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607687 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.607766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607739 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.608086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.608086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.608086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607901 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.608086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607933 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.608086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607956 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.608086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.607978 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.708705 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.708665 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.708705 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.708709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.708955 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.708729 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.708955 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.708938 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709117 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.708986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709117 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709117 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709054 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709117 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709348 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709348 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709348 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709199 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709348 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709348 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709348 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709438 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsfhk\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-kube-api-access-rsfhk\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709461 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.709639 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.709548 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.710488 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.710462 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.712653 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.712625 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.713555 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.713220 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.713555 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.713419 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.714475 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.713803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.714475 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.714174 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.714475 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.714185 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.714475 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.714179 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.714761 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.714744 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.715983 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.715952 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.716111 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.716089 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.716863 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.716841 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.717604 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.717578 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.718039 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.718012 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.718225 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.718201 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.718974 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.718955 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.719225 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.719200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.722993 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.722955 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsfhk\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-kube-api-access-rsfhk\") pod \"prometheus-k8s-0\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:51.844981 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.844887 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l2prr" Jul 20 16:46:51.859679 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:51.859643 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:46:52.072552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:52.072506 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:46:52.077388 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:46:52.077344 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea5386c_42b8_49a5_a0fa_82bc6b398093.slice/crio-013a7cb56cb5a6908debcfd9c98690dcf21cb2b1f4d2a7e28ee429b9515acba9 WatchSource:0}: Error finding container 013a7cb56cb5a6908debcfd9c98690dcf21cb2b1f4d2a7e28ee429b9515acba9: Status 404 returned error can't find the container with id 013a7cb56cb5a6908debcfd9c98690dcf21cb2b1f4d2a7e28ee429b9515acba9 Jul 20 16:46:53.015593 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:53.015551 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"013a7cb56cb5a6908debcfd9c98690dcf21cb2b1f4d2a7e28ee429b9515acba9"} Jul 20 16:46:55.025135 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:55.025017 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} Jul 20 16:46:56.030747 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:56.030711 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" exitCode=0 Jul 20 16:46:56.031150 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:46:56.030757 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} Jul 20 16:47:00.051426 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:00.051381 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} Jul 20 16:47:00.051849 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:00.051454 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} Jul 20 16:47:03.068597 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:03.068541 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} Jul 20 16:47:03.069029 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:03.068633 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} Jul 20 16:47:03.069029 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:03.068675 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} Jul 20 16:47:03.069029 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:03.068719 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerStarted","Data":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} Jul 20 16:47:03.109600 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:03.109537 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.380961264 podStartE2EDuration="12.109515836s" totalImagesPullingTime="8.708972333s" totalInitContainerRuntime="19.582239ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:46:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:47:03.107955232 +0000 UTC m=+112.122377527" watchObservedRunningTime="2026-07-20 16:47:03.109515836 +0000 UTC m=+112.123938092" Jul 20 16:47:06.860507 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:06.860463 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:47:10.394516 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.394453 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" podUID="bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" containerName="registry" containerID="cri-o://6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6" gracePeriod=30 Jul 20 16:47:10.679429 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.679359 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:47:10.792788 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792756 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-trusted-ca\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.792966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792823 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7drv9\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-kube-api-access-7drv9\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.792966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792851 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-image-registry-private-configuration\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.792966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792900 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.792966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792929 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-bound-sa-token\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.792966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792954 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-installation-pull-secrets\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.793233 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.792976 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-certificates\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.793233 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.793019 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-ca-trust-extracted\") pod \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\" (UID: \"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a\") " Jul 20 16:47:10.793369 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.793259 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:10.793651 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.793610 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:47:10.795728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.795698 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:10.795844 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.795740 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:47:10.795844 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.795816 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-kube-api-access-7drv9" (OuterVolumeSpecName: "kube-api-access-7drv9") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "kube-api-access-7drv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:47:10.795844 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.795833 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:47:10.796000 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.795944 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:47:10.801994 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.801973 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" (UID: "bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:47:10.894101 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894067 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-bound-sa-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894101 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894099 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-installation-pull-secrets\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894101 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894110 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-certificates\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894339 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894118 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-ca-trust-extracted\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894339 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894127 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-trusted-ca\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894339 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894135 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7drv9\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-kube-api-access-7drv9\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894339 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894145 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-image-registry-private-configuration\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:10.894339 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:10.894154 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a-registry-tls\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:47:11.101391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.101290 2579 generic.go:356] "Generic (PLEG): container finished" podID="bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" containerID="6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6" exitCode=0 Jul 20 16:47:11.101391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.101362 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" Jul 20 16:47:11.101562 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.101359 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" event={"ID":"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a","Type":"ContainerDied","Data":"6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6"} Jul 20 16:47:11.101562 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.101483 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c8dcd5578-dzmn4" event={"ID":"bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a","Type":"ContainerDied","Data":"a9d47434c43a26a7e18b4dead9a6bf6a0c365313ffb18e5f444de916a3b76a81"} Jul 20 16:47:11.101562 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.101508 2579 scope.go:117] "RemoveContainer" containerID="6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6" Jul 20 16:47:11.110436 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.110413 2579 scope.go:117] "RemoveContainer" containerID="6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6" Jul 20 16:47:11.110725 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:47:11.110702 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6\": container with ID starting with 6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6 not found: ID does not exist" containerID="6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6" Jul 20 16:47:11.110769 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.110734 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6"} err="failed to get container status \"6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6\": rpc error: code = NotFound desc = could not find container \"6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6\": container with ID starting with 6f1c8b7ef8d1d8d3d2a67249ee3e1aa0a0f7851dcef45e9884f2099eec4e78e6 not found: ID does not exist" Jul 20 16:47:11.122320 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.122269 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c8dcd5578-dzmn4"] Jul 20 16:47:11.126524 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.126497 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c8dcd5578-dzmn4"] Jul 20 16:47:11.473331 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:11.473288 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" path="/var/lib/kubelet/pods/bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a/volumes" Jul 20 16:47:19.132757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:19.132723 2579 generic.go:356] "Generic (PLEG): container finished" podID="e74d356e-e508-432c-a37c-e274dcd195b7" containerID="e672b5bd77391524b26f061c9bcd52df94331acca709bc6a9b5bc0324ff97d1b" exitCode=0 Jul 20 16:47:19.133163 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:19.132775 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" event={"ID":"e74d356e-e508-432c-a37c-e274dcd195b7","Type":"ContainerDied","Data":"e672b5bd77391524b26f061c9bcd52df94331acca709bc6a9b5bc0324ff97d1b"} Jul 20 16:47:19.133163 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:19.133100 2579 scope.go:117] "RemoveContainer" containerID="e672b5bd77391524b26f061c9bcd52df94331acca709bc6a9b5bc0324ff97d1b" Jul 20 16:47:20.138301 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:20.138262 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-dl6bh" event={"ID":"e74d356e-e508-432c-a37c-e274dcd195b7","Type":"ContainerStarted","Data":"cb09100d4be5da65794e7a3d339db736fb964470b47151ab8f97a7f5f1e14a8b"} Jul 20 16:47:24.153873 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:24.153840 2579 generic.go:356] "Generic (PLEG): container finished" podID="f09b4f5f-869c-422e-99e6-4b0df74d6733" containerID="e6fa07c328571a405ad0dd9c509853d6c44be05f704bcb5e745c6f6a4474a283" exitCode=0 Jul 20 16:47:24.154266 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:24.153920 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" event={"ID":"f09b4f5f-869c-422e-99e6-4b0df74d6733","Type":"ContainerDied","Data":"e6fa07c328571a405ad0dd9c509853d6c44be05f704bcb5e745c6f6a4474a283"} Jul 20 16:47:24.154332 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:24.154265 2579 scope.go:117] "RemoveContainer" containerID="e6fa07c328571a405ad0dd9c509853d6c44be05f704bcb5e745c6f6a4474a283" Jul 20 16:47:25.160236 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:25.160200 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-4hmr6" event={"ID":"f09b4f5f-869c-422e-99e6-4b0df74d6733","Type":"ContainerStarted","Data":"a6ae03c40005fed9799389a3038343d5e406c0edd634033c5189fd8a1e271ec3"} Jul 20 16:47:29.176212 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:29.176178 2579 generic.go:356] "Generic (PLEG): container finished" podID="87d0d3ff-2190-44b9-8b8b-833c9a0865cf" containerID="b92502f4f5636f3530734cc4241513be8b733e9508b2083a002593f17287bea1" exitCode=0 Jul 20 16:47:29.176603 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:29.176254 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" event={"ID":"87d0d3ff-2190-44b9-8b8b-833c9a0865cf","Type":"ContainerDied","Data":"b92502f4f5636f3530734cc4241513be8b733e9508b2083a002593f17287bea1"} Jul 20 16:47:29.176603 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:29.176582 2579 scope.go:117] "RemoveContainer" containerID="b92502f4f5636f3530734cc4241513be8b733e9508b2083a002593f17287bea1" Jul 20 16:47:30.182583 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:30.182550 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ljjp2" event={"ID":"87d0d3ff-2190-44b9-8b8b-833c9a0865cf","Type":"ContainerStarted","Data":"7ce097b28fa9f332ae6eafa0c9e2267ef29651e35a6afacc1b5a84df2dd010ac"} Jul 20 16:47:51.860785 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:51.860694 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:47:51.877026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:51.876998 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:47:52.285114 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:47:52.285079 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:09.861165 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861080 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:48:09.861728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861424 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="config-reloader" containerID="cri-o://8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" gracePeriod=600 Jul 20 16:48:09.861728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861412 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy" containerID="cri-o://cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" gracePeriod=600 Jul 20 16:48:09.861728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861446 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="thanos-sidecar" containerID="cri-o://7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" gracePeriod=600 Jul 20 16:48:09.861728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861468 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" gracePeriod=600 Jul 20 16:48:09.861728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861492 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-web" containerID="cri-o://c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" gracePeriod=600 Jul 20 16:48:09.861728 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:09.861401 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="prometheus" containerID="cri-o://1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" gracePeriod=600 Jul 20 16:48:10.114789 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.114732 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.264439 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264388 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-db\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264439 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264442 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-kubelet-serving-ca-bundle\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264469 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsfhk\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-kube-api-access-rsfhk\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264485 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-serving-certs-ca-bundle\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264502 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-kube-rbac-proxy\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264524 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264554 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-0\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264688 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264578 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-tls-assets\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.264980 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.264961 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-tls\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265038 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265000 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.265038 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265010 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-thanos-prometheus-http-client-file\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265153 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265041 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config-out\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265153 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265078 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-grpc-tls\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265153 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265107 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-web-config\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265153 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265141 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265376 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265170 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-2\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265376 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265193 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-metrics-client-ca\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265376 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265218 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-trusted-ca-bundle\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265376 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265248 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-1\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265744 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265457 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-metrics-client-certs\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265744 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265493 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config\") pod \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\" (UID: \"7ea5386c-42b8-49a5-a0fa-82bc6b398093\") " Jul 20 16:48:10.265744 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265564 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:10.265905 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265788 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-db\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.265905 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265809 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.266015 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.265936 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.266288 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.266259 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.267272 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.267240 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.267626 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.267601 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.268329 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.267811 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.268329 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.267896 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:48:10.268329 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.268099 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.268578 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.268551 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 16:48:10.268914 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.268891 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.269077 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.269051 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-kube-api-access-rsfhk" (OuterVolumeSpecName: "kube-api-access-rsfhk") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "kube-api-access-rsfhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 16:48:10.269077 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.269062 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.269186 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.269107 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config" (OuterVolumeSpecName: "config") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.269344 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.269297 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.269779 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.269762 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config-out" (OuterVolumeSpecName: "config-out") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 16:48:10.270171 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.270158 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.270439 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.270423 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.270955 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.270942 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.280920 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.280894 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-web-config" (OuterVolumeSpecName: "web-config") pod "7ea5386c-42b8-49a5-a0fa-82bc6b398093" (UID: "7ea5386c-42b8-49a5-a0fa-82bc6b398093"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 16:48:10.344599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344565 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" exitCode=0 Jul 20 16:48:10.344599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344588 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" exitCode=0 Jul 20 16:48:10.344599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344595 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" exitCode=0 Jul 20 16:48:10.344599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344602 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" exitCode=0 Jul 20 16:48:10.344599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344607 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" exitCode=0 Jul 20 16:48:10.344599 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344612 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" exitCode=0 Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344641 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344651 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344719 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344771 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344801 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344817 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} Jul 20 16:48:10.344879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.344840 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ea5386c-42b8-49a5-a0fa-82bc6b398093","Type":"ContainerDied","Data":"013a7cb56cb5a6908debcfd9c98690dcf21cb2b1f4d2a7e28ee429b9515acba9"} Jul 20 16:48:10.353048 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.353032 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.360196 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.360181 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.366385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366333 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-tls-assets\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366355 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366367 2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366376 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config-out\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366386 2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-grpc-tls\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366399 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-web-config\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366410 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366422 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366436 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-metrics-client-ca\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366448 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366461 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366473 2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-metrics-client-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366485 2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-config\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366497 2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366510 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsfhk\" (UniqueName: \"kubernetes.io/projected/7ea5386c-42b8-49a5-a0fa-82bc6b398093-kube-api-access-rsfhk\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366522 2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-kube-rbac-proxy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366545 2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ea5386c-42b8-49a5-a0fa-82bc6b398093-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.366587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366560 2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ea5386c-42b8-49a5-a0fa-82bc6b398093-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 16:48:10.367000 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.366948 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.373855 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.373834 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.380626 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.380598 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.385256 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.385234 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:48:10.388283 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.388261 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.397482 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.397461 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.398050 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.398030 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.398116 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398060 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} err="failed to get container status \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" Jul 20 16:48:10.398116 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398080 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.398334 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.398296 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.398391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398342 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} err="failed to get container status \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" Jul 20 16:48:10.398391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398362 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.398654 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.398634 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.398748 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398658 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} err="failed to get container status \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" Jul 20 16:48:10.398748 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398676 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.398949 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.398935 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.398992 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398953 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} err="failed to get container status \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" Jul 20 16:48:10.398992 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.398965 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.399107 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399086 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:48:10.399196 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.399175 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.399240 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399201 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} err="failed to get container status \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" Jul 20 16:48:10.399240 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399217 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.399471 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.399454 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.399530 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399474 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} err="failed to get container status \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" Jul 20 16:48:10.399530 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399489 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.399728 ip-10-0-131-206 kubenswrapper[2579]: E0720 16:48:10.399711 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.399772 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399732 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} err="failed to get container status \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" Jul 20 16:48:10.399772 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399747 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.399963 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399947 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} err="failed to get container status \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" Jul 20 16:48:10.400007 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.399964 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.400169 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400148 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} err="failed to get container status \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" Jul 20 16:48:10.400215 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400172 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.400419 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400397 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} err="failed to get container status \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" Jul 20 16:48:10.400465 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400421 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.400655 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400637 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} err="failed to get container status \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" Jul 20 16:48:10.400695 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400655 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.400860 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400845 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} err="failed to get container status \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" Jul 20 16:48:10.400906 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.400860 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.401074 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401057 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} err="failed to get container status \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" Jul 20 16:48:10.401114 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401075 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.401299 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401278 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} err="failed to get container status \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" Jul 20 16:48:10.401376 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401299 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.401539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401523 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} err="failed to get container status \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" Jul 20 16:48:10.401581 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401539 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.401748 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401723 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} err="failed to get container status \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" Jul 20 16:48:10.401804 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401748 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.401962 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401946 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} err="failed to get container status \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" Jul 20 16:48:10.402009 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.401962 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.402166 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402144 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} err="failed to get container status \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" Jul 20 16:48:10.402204 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402167 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.402411 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402393 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} err="failed to get container status \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" Jul 20 16:48:10.402503 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402412 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.402628 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402612 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} err="failed to get container status \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" Jul 20 16:48:10.402668 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402629 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.402836 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402818 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} err="failed to get container status \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" Jul 20 16:48:10.402916 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.402837 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.403046 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403029 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} err="failed to get container status \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" Jul 20 16:48:10.403093 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403046 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.403265 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403249 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} err="failed to get container status \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" Jul 20 16:48:10.403447 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403265 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.403503 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403470 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} err="failed to get container status \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" Jul 20 16:48:10.403503 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403482 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.403697 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403681 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} err="failed to get container status \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" Jul 20 16:48:10.403742 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403698 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.403908 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403893 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} err="failed to get container status \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" Jul 20 16:48:10.403953 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.403910 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.404100 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404081 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} err="failed to get container status \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" Jul 20 16:48:10.404100 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404098 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.404331 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404299 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} err="failed to get container status \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" Jul 20 16:48:10.404377 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404332 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.404543 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404527 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} err="failed to get container status \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" Jul 20 16:48:10.404609 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404544 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.404766 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404748 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} err="failed to get container status \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" Jul 20 16:48:10.404816 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404767 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.404983 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404966 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} err="failed to get container status \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" Jul 20 16:48:10.404983 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.404983 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.405189 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405174 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} err="failed to get container status \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" Jul 20 16:48:10.405227 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405189 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.405420 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405405 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} err="failed to get container status \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" Jul 20 16:48:10.405466 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405421 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.405624 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405604 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} err="failed to get container status \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" Jul 20 16:48:10.405664 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405624 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.405832 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405811 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} err="failed to get container status \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" Jul 20 16:48:10.405902 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.405833 2579 scope.go:117] "RemoveContainer" containerID="0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936" Jul 20 16:48:10.406053 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406029 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936"} err="failed to get container status \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": rpc error: code = NotFound desc = could not find container \"0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936\": container with ID starting with 0b9fcd9718f9d6f981933723f30a8fdaeff302318a2ccbf7a94a528a52484936 not found: ID does not exist" Jul 20 16:48:10.406053 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406045 2579 scope.go:117] "RemoveContainer" containerID="cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832" Jul 20 16:48:10.406246 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406231 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832"} err="failed to get container status \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": rpc error: code = NotFound desc = could not find container \"cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832\": container with ID starting with cea108f344c814e5e8edb7f87d84f3460656f1e5cd9dc0aed507f2064064c832 not found: ID does not exist" Jul 20 16:48:10.406300 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406247 2579 scope.go:117] "RemoveContainer" containerID="c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc" Jul 20 16:48:10.406464 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406447 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc"} err="failed to get container status \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": rpc error: code = NotFound desc = could not find container \"c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc\": container with ID starting with c0a683303f25ef95d7c59ce25e5203e8a8d1c211e72291ec0ea642a654d0dffc not found: ID does not exist" Jul 20 16:48:10.406502 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406465 2579 scope.go:117] "RemoveContainer" containerID="7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233" Jul 20 16:48:10.406652 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406637 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233"} err="failed to get container status \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": rpc error: code = NotFound desc = could not find container \"7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233\": container with ID starting with 7a9ad63cbf9154ae348ad9fd9277171ab7f50993389aaf8cb5c6e078a0529233 not found: ID does not exist" Jul 20 16:48:10.406697 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406652 2579 scope.go:117] "RemoveContainer" containerID="8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b" Jul 20 16:48:10.406861 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406840 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b"} err="failed to get container status \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": rpc error: code = NotFound desc = could not find container \"8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b\": container with ID starting with 8ded6158bd2ceab8d948d718489090a7bc2429fe0f62a96a374ddd554b83852b not found: ID does not exist" Jul 20 16:48:10.406861 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.406854 2579 scope.go:117] "RemoveContainer" containerID="1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185" Jul 20 16:48:10.407047 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.407026 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185"} err="failed to get container status \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": rpc error: code = NotFound desc = could not find container \"1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185\": container with ID starting with 1d7ddb0708900b4792b419546fa97f1475a50e9646012a1089815a1a3191d185 not found: ID does not exist" Jul 20 16:48:10.407047 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.407046 2579 scope.go:117] "RemoveContainer" containerID="5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153" Jul 20 16:48:10.407238 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.407215 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153"} err="failed to get container status \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": rpc error: code = NotFound desc = could not find container \"5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153\": container with ID starting with 5cdb35a5c2fa9b264fd44cd4e28211754ddcee82f80cd32ec9db9a1628963153 not found: ID does not exist" Jul 20 16:48:10.436818 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.436789 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:48:10.437063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437042 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="init-config-reloader" Jul 20 16:48:10.437063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437058 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="init-config-reloader" Jul 20 16:48:10.437063 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437068 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-web" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437073 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-web" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437080 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437086 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437094 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-thanos" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437099 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-thanos" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437108 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="thanos-sidecar" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437113 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="thanos-sidecar" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437121 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="config-reloader" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437126 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="config-reloader" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437133 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" containerName="registry" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437138 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" containerName="registry" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437144 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="prometheus" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437148 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="prometheus" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437191 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="prometheus" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437198 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="config-reloader" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437205 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="thanos-sidecar" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437210 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-web" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437217 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy-thanos" Jul 20 16:48:10.437211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437222 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" containerName="kube-rbac-proxy" Jul 20 16:48:10.437860 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.437227 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc272cd8-a7a5-4bad-aa0e-243b5c6cf99a" containerName="registry" Jul 20 16:48:10.440975 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.440960 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.445511 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.445491 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 20 16:48:10.445619 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.445527 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 20 16:48:10.445619 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.445526 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 20 16:48:10.445811 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.445796 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 20 16:48:10.445859 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.445803 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 20 16:48:10.445930 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.445842 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446797 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446832 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446873 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446881 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446892 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446906 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e5j2kiuj2hmto\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446921 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 20 16:48:10.447012 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.446802 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 16:48:10.447533 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.447515 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h6kkk\"" Jul 20 16:48:10.451601 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.451582 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 20 16:48:10.452692 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.452675 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 20 16:48:10.474057 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.474027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:48:10.568206 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmvl6\" (UniqueName: \"kubernetes.io/projected/438f4edb-a383-4879-b8dd-7173c6e41e80-kube-api-access-kmvl6\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568206 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568207 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-config\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568339 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/438f4edb-a383-4879-b8dd-7173c6e41e80-config-out\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568367 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568407 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568403 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568606 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568459 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568606 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568606 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568606 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568589 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568606 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568604 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568629 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-web-config\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568646 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568696 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/438f4edb-a383-4879-b8dd-7173c6e41e80-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568748 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.568914 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.568793 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669695 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669695 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669747 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-web-config\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669845 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.669966 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.669875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/438f4edb-a383-4879-b8dd-7173c6e41e80-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.670492 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670464 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.670670 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670488 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.670670 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670579 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.670833 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.670833 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670718 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.670833 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670757 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671003 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670835 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmvl6\" (UniqueName: \"kubernetes.io/projected/438f4edb-a383-4879-b8dd-7173c6e41e80-kube-api-access-kmvl6\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671003 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.670918 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671379 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671358 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671531 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-config\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671642 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671627 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671745 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/438f4edb-a383-4879-b8dd-7173c6e41e80-config-out\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671838 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671825 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.671938 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671924 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.672035 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.672021 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.672147 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.672132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.673168 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.673142 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/438f4edb-a383-4879-b8dd-7173c6e41e80-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.673257 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.673212 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.673386 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.673365 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-web-config\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.673628 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.673707 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.673713 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.671823 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.674464 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.674496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-config\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.674539 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.674512 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/438f4edb-a383-4879-b8dd-7173c6e41e80-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.675300 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.675277 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.675933 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.675905 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.675933 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.675920 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/438f4edb-a383-4879-b8dd-7173c6e41e80-config-out\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.676091 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.675954 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.676242 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.676222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/438f4edb-a383-4879-b8dd-7173c6e41e80-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.680475 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.680458 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmvl6\" (UniqueName: \"kubernetes.io/projected/438f4edb-a383-4879-b8dd-7173c6e41e80-kube-api-access-kmvl6\") pod \"prometheus-k8s-0\" (UID: \"438f4edb-a383-4879-b8dd-7173c6e41e80\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.750387 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.750343 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:48:10.898656 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:10.898627 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 20 16:48:10.902201 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:48:10.902175 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438f4edb_a383_4879_b8dd_7173c6e41e80.slice/crio-7b78b52fac814f40fed497f694a00015ef9ba84c46bc6fa77af21adee00e3906 WatchSource:0}: Error finding container 7b78b52fac814f40fed497f694a00015ef9ba84c46bc6fa77af21adee00e3906: Status 404 returned error can't find the container with id 7b78b52fac814f40fed497f694a00015ef9ba84c46bc6fa77af21adee00e3906 Jul 20 16:48:11.349601 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:11.349570 2579 generic.go:356] "Generic (PLEG): container finished" podID="438f4edb-a383-4879-b8dd-7173c6e41e80" containerID="ac8a4b212d25e3e6a9bb55fdf5eaf5bbee3968a65c8812f76141cd8d04a45917" exitCode=0 Jul 20 16:48:11.349745 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:11.349612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerDied","Data":"ac8a4b212d25e3e6a9bb55fdf5eaf5bbee3968a65c8812f76141cd8d04a45917"} Jul 20 16:48:11.349745 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:11.349698 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"7b78b52fac814f40fed497f694a00015ef9ba84c46bc6fa77af21adee00e3906"} Jul 20 16:48:11.475956 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:11.475930 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ea5386c-42b8-49a5-a0fa-82bc6b398093" path="/var/lib/kubelet/pods/7ea5386c-42b8-49a5-a0fa-82bc6b398093/volumes" Jul 20 16:48:12.356585 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.356498 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"a718c07ebb0f1a48e61706117f4d6cf5e1e12246c0cd4cda3783f6b50216264b"} Jul 20 16:48:12.356585 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.356564 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"8d436aef1227b401843d90696833a743a814f5e284821dfe7892e41741e5e92c"} Jul 20 16:48:12.357080 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.356592 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"6705330ac32a1aba567ea9df440326d378922918cba1dc3a6944745237918b0d"} Jul 20 16:48:12.357080 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.356615 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"d3cb8eb4efb680396b14af2bf6b4f4b442293cd95d47f215b0abcd951dc67d6f"} Jul 20 16:48:12.357080 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.356638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"60d1fd8bdc85dc10585bf4e6cbf9705f45025e0061669eff9bbd96536c5f1499"} Jul 20 16:48:12.357080 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.356669 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"438f4edb-a383-4879-b8dd-7173c6e41e80","Type":"ContainerStarted","Data":"c87b6e01328d227bb2012f2226669b101f9bf55be7344a19339ae396fa5d8167"} Jul 20 16:48:12.393072 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:12.393006 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.3736606780000002 podStartE2EDuration="2.392985936s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.325258ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:48:12.390487242 +0000 UTC m=+181.404909497" watchObservedRunningTime="2026-07-20 16:48:12.392985936 +0000 UTC m=+181.407408200" Jul 20 16:48:15.751424 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:48:15.751383 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:49:10.751263 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:49:10.751205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:49:10.767113 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:49:10.767087 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:49:11.599963 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:49:11.599936 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 20 16:50:11.323391 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:11.323362 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:50:11.328086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:11.328062 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:50:11.328282 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:11.328267 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:50:11.333382 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:11.333356 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:50:11.335525 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:11.335496 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 16:50:16.614337 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.612977 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt"] Jul 20 16:50:16.616934 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.616909 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.625530 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.625504 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 16:50:16.625653 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.625533 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 16:50:16.625653 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.625558 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zwkbp\"" Jul 20 16:50:16.625653 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.625591 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 16:50:16.625653 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.625608 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 16:50:16.639632 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.639607 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt"] Jul 20 16:50:16.698217 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.698180 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeb0e651-9ed6-4579-bb42-58b549be40d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.698414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.698226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eeb0e651-9ed6-4579-bb42-58b549be40d1-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.698414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.698289 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eeb0e651-9ed6-4579-bb42-58b549be40d1-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.698414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.698349 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eeb0e651-9ed6-4579-bb42-58b549be40d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.698414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.698380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jr4d\" (UniqueName: \"kubernetes.io/projected/eeb0e651-9ed6-4579-bb42-58b549be40d1-kube-api-access-8jr4d\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.799095 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.799060 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eeb0e651-9ed6-4579-bb42-58b549be40d1-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.799288 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.799117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eeb0e651-9ed6-4579-bb42-58b549be40d1-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.799288 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.799137 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eeb0e651-9ed6-4579-bb42-58b549be40d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.799288 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.799163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jr4d\" (UniqueName: \"kubernetes.io/projected/eeb0e651-9ed6-4579-bb42-58b549be40d1-kube-api-access-8jr4d\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.799288 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.799205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeb0e651-9ed6-4579-bb42-58b549be40d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.799646 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.799609 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eeb0e651-9ed6-4579-bb42-58b549be40d1-manifests\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.801572 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.801538 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eeb0e651-9ed6-4579-bb42-58b549be40d1-tmp\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.801706 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.801686 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eeb0e651-9ed6-4579-bb42-58b549be40d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.801761 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.801738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeb0e651-9ed6-4579-bb42-58b549be40d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.829919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.829888 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jr4d\" (UniqueName: \"kubernetes.io/projected/eeb0e651-9ed6-4579-bb42-58b549be40d1-kube-api-access-8jr4d\") pod \"opendatahub-operator-controller-manager-76b8cd99f7-7jsnt\" (UID: \"eeb0e651-9ed6-4579-bb42-58b549be40d1\") " pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:16.927214 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:16.927175 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:17.101402 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:17.101367 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt"] Jul 20 16:50:17.126171 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:50:17.126142 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb0e651_9ed6_4579_bb42_58b549be40d1.slice/crio-b29ef2aad846675861452524e276718e17607bb7b49713b66914e576df15cac7 WatchSource:0}: Error finding container b29ef2aad846675861452524e276718e17607bb7b49713b66914e576df15cac7: Status 404 returned error can't find the container with id b29ef2aad846675861452524e276718e17607bb7b49713b66914e576df15cac7 Jul 20 16:50:17.127948 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:17.127927 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:50:17.832385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:17.832348 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" event={"ID":"eeb0e651-9ed6-4579-bb42-58b549be40d1","Type":"ContainerStarted","Data":"b29ef2aad846675861452524e276718e17607bb7b49713b66914e576df15cac7"} Jul 20 16:50:20.847825 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:20.847780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" event={"ID":"eeb0e651-9ed6-4579-bb42-58b549be40d1","Type":"ContainerStarted","Data":"e6e3aece3c3f5256bc8dd1f994caef99cf83f4fe917f530efbfba14be20053c9"} Jul 20 16:50:21.854202 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:21.854166 2579 generic.go:356] "Generic (PLEG): container finished" podID="eeb0e651-9ed6-4579-bb42-58b549be40d1" containerID="e6e3aece3c3f5256bc8dd1f994caef99cf83f4fe917f530efbfba14be20053c9" exitCode=0 Jul 20 16:50:21.854658 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:21.854251 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" event={"ID":"eeb0e651-9ed6-4579-bb42-58b549be40d1","Type":"ContainerDied","Data":"e6e3aece3c3f5256bc8dd1f994caef99cf83f4fe917f530efbfba14be20053c9"} Jul 20 16:50:22.860610 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:22.860575 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" event={"ID":"eeb0e651-9ed6-4579-bb42-58b549be40d1","Type":"ContainerStarted","Data":"0e58d92e3d7144db628a14e4332c19254afa671747d4b7876b814abbff95d7be"} Jul 20 16:50:22.861060 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:22.860697 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:33.867288 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:33.867253 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" Jul 20 16:50:33.898286 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:33.898231 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76b8cd99f7-7jsnt" podStartSLOduration=13.560845381 podStartE2EDuration="17.898213344s" totalImagesPullingTime="3.335281387s" totalInitContainerRuntime="1.002086576s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:22.900468105 +0000 UTC m=+311.914890361" watchObservedRunningTime="2026-07-20 16:50:33.898213344 +0000 UTC m=+322.912635599" Jul 20 16:50:35.965056 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.965023 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86568699db-l5zbj"] Jul 20 16:50:35.971495 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.971477 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:35.977828 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.977788 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 16:50:35.979053 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.979010 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 16:50:35.979187 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.979036 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-49kdd\"" Jul 20 16:50:35.979187 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.979179 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 16:50:35.979325 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.979203 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 16:50:35.980112 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:35.980098 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 16:50:36.015071 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.015033 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86568699db-l5zbj"] Jul 20 16:50:36.062985 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.062954 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6784b9bd-7e60-4597-8b18-291d09216022-manager-config\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.063154 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.063002 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6784b9bd-7e60-4597-8b18-291d09216022-metrics-cert\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.063154 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.063050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5pg8\" (UniqueName: \"kubernetes.io/projected/6784b9bd-7e60-4597-8b18-291d09216022-kube-api-access-f5pg8\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.063154 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.063116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6784b9bd-7e60-4597-8b18-291d09216022-cert\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.163825 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.163788 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5pg8\" (UniqueName: \"kubernetes.io/projected/6784b9bd-7e60-4597-8b18-291d09216022-kube-api-access-f5pg8\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.164007 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.163904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6784b9bd-7e60-4597-8b18-291d09216022-cert\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.164007 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.163960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6784b9bd-7e60-4597-8b18-291d09216022-manager-config\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.164007 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.164001 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6784b9bd-7e60-4597-8b18-291d09216022-metrics-cert\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.164726 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.164705 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6784b9bd-7e60-4597-8b18-291d09216022-manager-config\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.166750 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.166723 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6784b9bd-7e60-4597-8b18-291d09216022-cert\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.166750 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.166742 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6784b9bd-7e60-4597-8b18-291d09216022-metrics-cert\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.186480 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.186449 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5pg8\" (UniqueName: \"kubernetes.io/projected/6784b9bd-7e60-4597-8b18-291d09216022-kube-api-access-f5pg8\") pod \"lws-controller-manager-86568699db-l5zbj\" (UID: \"6784b9bd-7e60-4597-8b18-291d09216022\") " pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.280127 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.280020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:36.440435 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.440411 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86568699db-l5zbj"] Jul 20 16:50:36.442502 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:50:36.442470 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6784b9bd_7e60_4597_8b18_291d09216022.slice/crio-3a3c2f7c0e135ac30ab8dec87c979309143e8664c83ed1d25713404572a0affc WatchSource:0}: Error finding container 3a3c2f7c0e135ac30ab8dec87c979309143e8664c83ed1d25713404572a0affc: Status 404 returned error can't find the container with id 3a3c2f7c0e135ac30ab8dec87c979309143e8664c83ed1d25713404572a0affc Jul 20 16:50:36.916541 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:36.916506 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" event={"ID":"6784b9bd-7e60-4597-8b18-291d09216022","Type":"ContainerStarted","Data":"3a3c2f7c0e135ac30ab8dec87c979309143e8664c83ed1d25713404572a0affc"} Jul 20 16:50:38.929332 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:38.929216 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" event={"ID":"6784b9bd-7e60-4597-8b18-291d09216022","Type":"ContainerStarted","Data":"b80c60141da69e3fcbe6ae621474db63864eb1625c7bdf54cbf77012ca847689"} Jul 20 16:50:38.929332 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:38.929299 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:38.979003 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:38.978946 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" podStartSLOduration=1.7855397370000001 podStartE2EDuration="3.978929552s" totalImagesPullingTime="2.193389815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:38.977071203 +0000 UTC m=+327.991493470" watchObservedRunningTime="2026-07-20 16:50:38.978929552 +0000 UTC m=+327.993351890" Jul 20 16:50:48.577248 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.577214 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74"] Jul 20 16:50:48.580607 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.580591 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.587140 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.586848 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 16:50:48.587230 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.587164 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bcgp7\"" Jul 20 16:50:48.673299 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673259 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673346 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jc9p\" (UniqueName: \"kubernetes.io/projected/29a0099a-00f5-4c62-ad2f-b340fd793176-kube-api-access-7jc9p\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673397 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673424 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29a0099a-00f5-4c62-ad2f-b340fd793176-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673542 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673559 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.673756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.673645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.774610 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.774610 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.774862 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jc9p\" (UniqueName: \"kubernetes.io/projected/29a0099a-00f5-4c62-ad2f-b340fd793176-kube-api-access-7jc9p\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.774862 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.774862 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775027 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774872 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29a0099a-00f5-4c62-ad2f-b340fd793176-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775027 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774923 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775027 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774959 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775027 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.774997 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.775063 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775211 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.775180 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775289 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.775269 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775387 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.775359 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.775657 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.775638 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29a0099a-00f5-4c62-ad2f-b340fd793176-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.777412 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.777388 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.777568 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.777552 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.795281 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.795260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jc9p\" (UniqueName: \"kubernetes.io/projected/29a0099a-00f5-4c62-ad2f-b340fd793176-kube-api-access-7jc9p\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.817022 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.816994 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29a0099a-00f5-4c62-ad2f-b340fd793176-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zzb74\" (UID: \"29a0099a-00f5-4c62-ad2f-b340fd793176\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:48.823843 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.823819 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74"] Jul 20 16:50:48.890645 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:48.890618 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:49.107987 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:49.107943 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74"] Jul 20 16:50:49.115617 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:50:49.115592 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a0099a_00f5_4c62_ad2f_b340fd793176.slice/crio-4070276c5bfce54e85ac4ef350247d00a3a65edbcf69cd45d743510150d0949d WatchSource:0}: Error finding container 4070276c5bfce54e85ac4ef350247d00a3a65edbcf69cd45d743510150d0949d: Status 404 returned error can't find the container with id 4070276c5bfce54e85ac4ef350247d00a3a65edbcf69cd45d743510150d0949d Jul 20 16:50:49.935576 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:49.935530 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86568699db-l5zbj" Jul 20 16:50:49.984925 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:49.984882 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" event={"ID":"29a0099a-00f5-4c62-ad2f-b340fd793176","Type":"ContainerStarted","Data":"4070276c5bfce54e85ac4ef350247d00a3a65edbcf69cd45d743510150d0949d"} Jul 20 16:50:51.045780 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.045747 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p"] Jul 20 16:50:51.053053 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.053026 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.058680 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.058624 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 20 16:50:51.058680 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.058644 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 20 16:50:51.058897 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.058634 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 20 16:50:51.058897 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.058745 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lffvv\"" Jul 20 16:50:51.086453 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.086419 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p"] Jul 20 16:50:51.093940 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.093906 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfea2583-c0b7-4473-933c-02d489624373-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.094093 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.093966 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.094093 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.093991 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cfea2583-c0b7-4473-933c-02d489624373-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.094093 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.094069 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfea2583-c0b7-4473-933c-02d489624373-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.094261 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.094095 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.094261 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.094130 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.094261 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.094183 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czr9g\" (UniqueName: \"kubernetes.io/projected/cfea2583-c0b7-4473-933c-02d489624373-kube-api-access-czr9g\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195294 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfea2583-c0b7-4473-933c-02d489624373-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195294 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195339 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cfea2583-c0b7-4473-933c-02d489624373-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfea2583-c0b7-4473-933c-02d489624373-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195435 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.195552 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.195475 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czr9g\" (UniqueName: \"kubernetes.io/projected/cfea2583-c0b7-4473-933c-02d489624373-kube-api-access-czr9g\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.196183 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.196126 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cfea2583-c0b7-4473-933c-02d489624373-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.198443 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.198419 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.198558 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.198488 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfea2583-c0b7-4473-933c-02d489624373-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.198620 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.198572 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.198996 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.198961 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cfea2583-c0b7-4473-933c-02d489624373-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.206967 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.206931 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfea2583-c0b7-4473-933c-02d489624373-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.207414 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.207388 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czr9g\" (UniqueName: \"kubernetes.io/projected/cfea2583-c0b7-4473-933c-02d489624373-kube-api-access-czr9g\") pod \"istiod-openshift-gateway-94bb8fbfd-brv2p\" (UID: \"cfea2583-c0b7-4473-933c-02d489624373\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.364828 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.364740 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:51.838600 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.838544 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:50:51.838711 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.838657 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:50:51.838757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.838709 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:50:51.983631 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.983596 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p"] Jul 20 16:50:51.988843 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:50:51.988815 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfea2583_c0b7_4473_933c_02d489624373.slice/crio-cfda722aae7fbc89a16f21389a1ddd881fbfd4222614459b148ff5f6e4bc7c35 WatchSource:0}: Error finding container cfda722aae7fbc89a16f21389a1ddd881fbfd4222614459b148ff5f6e4bc7c35: Status 404 returned error can't find the container with id cfda722aae7fbc89a16f21389a1ddd881fbfd4222614459b148ff5f6e4bc7c35 Jul 20 16:50:51.995937 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.995910 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" event={"ID":"cfea2583-c0b7-4473-933c-02d489624373","Type":"ContainerStarted","Data":"cfda722aae7fbc89a16f21389a1ddd881fbfd4222614459b148ff5f6e4bc7c35"} Jul 20 16:50:51.997674 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:51.997637 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" event={"ID":"29a0099a-00f5-4c62-ad2f-b340fd793176","Type":"ContainerStarted","Data":"189e92fc2885ed5c6c6113c0f39dc11f1e06db4b0e226ad54b1b990dc67877de"} Jul 20 16:50:52.072429 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:52.072320 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" podStartSLOduration=1.3517757160000001 podStartE2EDuration="4.072279382s" totalImagesPullingTime="2.720503666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:52.066340749 +0000 UTC m=+341.080763031" watchObservedRunningTime="2026-07-20 16:50:52.072279382 +0000 UTC m=+341.086701640" Jul 20 16:50:52.891430 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:52.891365 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:52.897458 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:52.897431 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:53.002711 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:53.002676 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:53.003792 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:53.003766 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zzb74" Jul 20 16:50:54.145216 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:54.145182 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:50:54.145548 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:54.145245 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:50:55.014633 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:55.014597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" event={"ID":"cfea2583-c0b7-4473-933c-02d489624373","Type":"ContainerStarted","Data":"24f8ed600993e7b57f3ed42ea781f8927c774af10e3668d692a22becc479d927"} Jul 20 16:50:55.014843 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:55.014673 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:50:55.016465 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:55.016441 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-brv2p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 16:50:55.016590 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:55.016484 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" podUID="cfea2583-c0b7-4473-933c-02d489624373" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 16:50:55.165760 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:55.164707 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" podStartSLOduration=3.011202302 podStartE2EDuration="5.164687082s" totalImagesPullingTime="2.15348478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:50:55.163253744 +0000 UTC m=+344.177676000" watchObservedRunningTime="2026-07-20 16:50:55.164687082 +0000 UTC m=+344.179109340" Jul 20 16:50:56.020466 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:50:56.020434 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-brv2p" Jul 20 16:53:15.348463 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.348427 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d"] Jul 20 16:53:15.350919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.350896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.358822 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.358804 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8fhss\"" Jul 20 16:53:15.368624 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.368596 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d"] Jul 20 16:53:15.448756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448719 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.448756 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448795 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448857 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4fch\" (UniqueName: \"kubernetes.io/projected/aaff3519-18e0-46e8-93f0-d1f8503fb069-kube-api-access-s4fch\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448902 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaff3519-18e0-46e8-93f0-d1f8503fb069-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.449041 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.448942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549569 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549569 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549817 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549817 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549817 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549979 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549979 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549879 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4fch\" (UniqueName: \"kubernetes.io/projected/aaff3519-18e0-46e8-93f0-d1f8503fb069-kube-api-access-s4fch\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549979 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.549979 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.549932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaff3519-18e0-46e8-93f0-d1f8503fb069-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.550261 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.550232 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.550349 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.550281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.550349 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.550282 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.550538 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.550402 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.550651 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.550630 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaff3519-18e0-46e8-93f0-d1f8503fb069-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.552142 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.552119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.552246 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.552227 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.558723 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.558695 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4fch\" (UniqueName: \"kubernetes.io/projected/aaff3519-18e0-46e8-93f0-d1f8503fb069-kube-api-access-s4fch\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.561430 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.561412 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaff3519-18e0-46e8-93f0-d1f8503fb069-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-px57d\" (UID: \"aaff3519-18e0-46e8-93f0-d1f8503fb069\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.661501 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.661471 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:15.800801 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.800777 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d"] Jul 20 16:53:15.802846 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:53:15.802819 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaff3519_18e0_46e8_93f0_d1f8503fb069.slice/crio-ab5ece8888f1b0bbf744e19e7e8112962f4c68966f37e6712e1fc25eac3c363e WatchSource:0}: Error finding container ab5ece8888f1b0bbf744e19e7e8112962f4c68966f37e6712e1fc25eac3c363e: Status 404 returned error can't find the container with id ab5ece8888f1b0bbf744e19e7e8112962f4c68966f37e6712e1fc25eac3c363e Jul 20 16:53:15.804856 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.804826 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:53:15.804919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.804888 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:53:15.804919 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:15.804914 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:53:16.630385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:16.630345 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" event={"ID":"aaff3519-18e0-46e8-93f0-d1f8503fb069","Type":"ContainerStarted","Data":"4b0538d0c7865245bbcf37386270c2aebbd97cb5e77f815a0d19e998e263dd11"} Jul 20 16:53:16.630774 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:16.630441 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" event={"ID":"aaff3519-18e0-46e8-93f0-d1f8503fb069","Type":"ContainerStarted","Data":"ab5ece8888f1b0bbf744e19e7e8112962f4c68966f37e6712e1fc25eac3c363e"} Jul 20 16:53:16.658084 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:16.658033 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podStartSLOduration=1.658015818 podStartE2EDuration="1.658015818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:16.65622456 +0000 UTC m=+485.670646817" watchObservedRunningTime="2026-07-20 16:53:16.658015818 +0000 UTC m=+485.672438134" Jul 20 16:53:16.662015 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:16.661992 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:16.663902 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:16.663874 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-px57d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body= Jul 20 16:53:16.664000 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:16.663936 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podUID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" Jul 20 16:53:17.665705 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:17.665677 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:18.347359 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.347323 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:53:18.350526 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.350503 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.353961 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.353939 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 20 16:53:18.354086 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.353986 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h9v4b\"" Jul 20 16:53:18.355413 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.355394 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 16:53:18.355507 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.355451 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 16:53:18.365545 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.365525 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:53:18.396925 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.396887 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:53:18.475912 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.475875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c590f4c9-16b8-42b2-acf3-709f7813e6ee-config-file\") pod \"limitador-limitador-54d6d988d5-7fkl7\" (UID: \"c590f4c9-16b8-42b2-acf3-709f7813e6ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.475912 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.475917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq6sk\" (UniqueName: \"kubernetes.io/projected/c590f4c9-16b8-42b2-acf3-709f7813e6ee-kube-api-access-qq6sk\") pod \"limitador-limitador-54d6d988d5-7fkl7\" (UID: \"c590f4c9-16b8-42b2-acf3-709f7813e6ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.576824 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.576786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c590f4c9-16b8-42b2-acf3-709f7813e6ee-config-file\") pod \"limitador-limitador-54d6d988d5-7fkl7\" (UID: \"c590f4c9-16b8-42b2-acf3-709f7813e6ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.577034 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.576834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq6sk\" (UniqueName: \"kubernetes.io/projected/c590f4c9-16b8-42b2-acf3-709f7813e6ee-kube-api-access-qq6sk\") pod \"limitador-limitador-54d6d988d5-7fkl7\" (UID: \"c590f4c9-16b8-42b2-acf3-709f7813e6ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.577623 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.577604 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c590f4c9-16b8-42b2-acf3-709f7813e6ee-config-file\") pod \"limitador-limitador-54d6d988d5-7fkl7\" (UID: \"c590f4c9-16b8-42b2-acf3-709f7813e6ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.586769 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.586733 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq6sk\" (UniqueName: \"kubernetes.io/projected/c590f4c9-16b8-42b2-acf3-709f7813e6ee-kube-api-access-qq6sk\") pod \"limitador-limitador-54d6d988d5-7fkl7\" (UID: \"c590f4c9-16b8-42b2-acf3-709f7813e6ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.639234 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.639205 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:18.640294 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.640277 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:53:18.660679 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.660651 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:18.815621 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:18.815500 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:53:18.817817 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:53:18.817791 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc590f4c9_16b8_42b2_acf3_709f7813e6ee.slice/crio-6948eca18a5d5415f71f4ef5a02105d03f1dde3055992477b2783530c53926f0 WatchSource:0}: Error finding container 6948eca18a5d5415f71f4ef5a02105d03f1dde3055992477b2783530c53926f0: Status 404 returned error can't find the container with id 6948eca18a5d5415f71f4ef5a02105d03f1dde3055992477b2783530c53926f0 Jul 20 16:53:19.645412 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:19.645376 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" event={"ID":"c590f4c9-16b8-42b2-acf3-709f7813e6ee","Type":"ContainerStarted","Data":"6948eca18a5d5415f71f4ef5a02105d03f1dde3055992477b2783530c53926f0"} Jul 20 16:53:21.663813 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:21.663780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" event={"ID":"c590f4c9-16b8-42b2-acf3-709f7813e6ee","Type":"ContainerStarted","Data":"b4a1675e942b5fc071abdb7d6b3b78086522f4dc59cbe465f00a7ccec2139db3"} Jul 20 16:53:21.664236 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:21.663874 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:53:21.684534 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:21.684487 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" podStartSLOduration=1.288892092 podStartE2EDuration="3.684472811s" totalImagesPullingTime="2.395580719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:21.684121926 +0000 UTC m=+490.698544184" watchObservedRunningTime="2026-07-20 16:53:21.684472811 +0000 UTC m=+490.698895070" Jul 20 16:53:32.669612 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:53:32.669584 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7fkl7" Jul 20 16:54:27.938569 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:27.938537 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:54:34.680533 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:34.680489 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-px57d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 16:54:34.680941 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:34.680566 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podUID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 16:54:35.992500 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:35.992468 2579 generic.go:356] "Generic (PLEG): container finished" podID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerID="4b0538d0c7865245bbcf37386270c2aebbd97cb5e77f815a0d19e998e263dd11" exitCode=0 Jul 20 16:54:35.992904 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:35.992540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" event={"ID":"aaff3519-18e0-46e8-93f0-d1f8503fb069","Type":"ContainerDied","Data":"4b0538d0c7865245bbcf37386270c2aebbd97cb5e77f815a0d19e998e263dd11"} Jul 20 16:54:35.992949 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:35.992927 2579 scope.go:117] "RemoveContainer" containerID="4b0538d0c7865245bbcf37386270c2aebbd97cb5e77f815a0d19e998e263dd11" Jul 20 16:54:35.993997 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:35.993970 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:54:35.994056 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:35.994043 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:54:35.994089 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:35.994076 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 16:54:37.002054 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:37.002009 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" event={"ID":"aaff3519-18e0-46e8-93f0-d1f8503fb069","Type":"ContainerStarted","Data":"c846e3a21c1915e6a99a5d5375e909e1a4a7e035cfae82e236ccb8400cfc2432"} Jul 20 16:54:37.662156 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:37.662116 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:54:37.662388 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:37.662363 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-px57d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body= Jul 20 16:54:37.662451 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:37.662433 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podUID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" Jul 20 16:54:38.661764 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:38.661728 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-px57d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body= Jul 20 16:54:38.662160 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:38.661781 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podUID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" Jul 20 16:54:39.662800 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:39.662765 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-px57d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body= Jul 20 16:54:39.663184 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:39.662821 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podUID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" Jul 20 16:54:40.662907 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:40.662861 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-px57d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body= Jul 20 16:54:40.663441 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:40.662945 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" podUID="aaff3519-18e0-46e8-93f0-d1f8503fb069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" Jul 20 16:54:41.664941 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:41.664901 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:54:41.665385 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:41.665179 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:54:41.666105 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:54:41.666084 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-px57d" Jul 20 16:55:00.700294 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:00.700254 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:55:11.352049 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:11.352016 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:55:11.356917 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:11.356895 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:55:11.361750 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:11.361732 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 16:55:11.366187 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:11.366171 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 16:55:15.624591 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:15.624554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:55:29.332917 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:29.332880 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:55:30.698409 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.698371 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq"] Jul 20 16:55:30.701170 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.701151 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.706130 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.706102 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 16:55:30.706280 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.706132 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 20 16:55:30.706280 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.706235 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 16:55:30.706280 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.706245 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k6shf\"" Jul 20 16:55:30.713684 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.713658 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq"] Jul 20 16:55:30.727026 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.726997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.727207 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.727052 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.727207 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.727150 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e9972a-4471-4d58-b602-4c25f727628d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.727207 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.727198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.727379 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.727235 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbngr\" (UniqueName: \"kubernetes.io/projected/78e9972a-4471-4d58-b602-4c25f727628d-kube-api-access-qbngr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.727379 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.727286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828229 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828469 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828250 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e9972a-4471-4d58-b602-4c25f727628d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828469 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828275 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828469 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbngr\" (UniqueName: \"kubernetes.io/projected/78e9972a-4471-4d58-b602-4c25f727628d-kube-api-access-qbngr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828469 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828702 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828510 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828757 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828710 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828792 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828752 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.828879 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.828863 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.830754 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.830729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78e9972a-4471-4d58-b602-4c25f727628d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.831107 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.831087 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78e9972a-4471-4d58-b602-4c25f727628d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:30.838330 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:30.838279 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbngr\" (UniqueName: \"kubernetes.io/projected/78e9972a-4471-4d58-b602-4c25f727628d-kube-api-access-qbngr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq\" (UID: \"78e9972a-4471-4d58-b602-4c25f727628d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:31.011983 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:31.011904 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:31.150857 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:31.150818 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq"] Jul 20 16:55:31.154200 ip-10-0-131-206 kubenswrapper[2579]: W0720 16:55:31.154173 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e9972a_4471_4d58_b602_4c25f727628d.slice/crio-7e9cd33308bcb8e27dbfe295a83c532b4fdd9c806b77c7d68ae17ed5e926c7d0 WatchSource:0}: Error finding container 7e9cd33308bcb8e27dbfe295a83c532b4fdd9c806b77c7d68ae17ed5e926c7d0: Status 404 returned error can't find the container with id 7e9cd33308bcb8e27dbfe295a83c532b4fdd9c806b77c7d68ae17ed5e926c7d0 Jul 20 16:55:31.155739 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:31.155720 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 16:55:31.266179 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:31.266097 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" event={"ID":"78e9972a-4471-4d58-b602-4c25f727628d","Type":"ContainerStarted","Data":"7e9cd33308bcb8e27dbfe295a83c532b4fdd9c806b77c7d68ae17ed5e926c7d0"} Jul 20 16:55:32.272545 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:32.272465 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" event={"ID":"78e9972a-4471-4d58-b602-4c25f727628d","Type":"ContainerStarted","Data":"c7b802623a1512c8a66d553482e22c0c70974f54c76b7fa57cae681144a89cee"} Jul 20 16:55:32.298800 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:32.298745 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" podStartSLOduration=1.292205521 podStartE2EDuration="2.298726825s" totalImagesPullingTime="1.006521304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:55:32.295691861 +0000 UTC m=+621.310114138" watchObservedRunningTime="2026-07-20 16:55:32.298726825 +0000 UTC m=+621.313149080" Jul 20 16:55:36.205347 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:36.205291 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:55:40.919010 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:40.918972 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:55:41.012679 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:41.012645 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:41.012857 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:41.012719 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:41.025587 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:41.025556 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:41.324286 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:41.324204 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq" Jul 20 16:55:52.440494 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:55:52.440457 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:57:49.625362 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:57:49.625319 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:57:58.711384 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:57:58.711342 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:58:10.532137 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:58:10.532102 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 16:59:52.813655 ip-10-0-131-206 kubenswrapper[2579]: I0720 16:59:52.813614 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:00:01.500133 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:01.500093 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:00:11.380943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:11.380856 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:00:11.385993 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:11.385970 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:00:11.387941 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:11.387921 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:00:11.393338 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:11.393291 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:00:34.304940 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:34.304903 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:00:43.705650 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:43.705616 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:00:52.211744 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:00:52.211704 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:01:01.406789 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:01:01.406754 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:01:11.705295 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:01:11.705260 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:01:21.806133 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:01:21.806095 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:01:32.599968 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:01:32.599881 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:01:42.233017 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:01:42.232975 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:02:04.805383 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:02:04.805350 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:02:14.705768 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:02:14.705726 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:02:38.306076 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:02:38.306033 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:02:48.899696 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:02:48.899660 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:03:04.306139 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:03:04.306060 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:03:14.151517 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:03:14.151478 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:03:23.403155 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:03:23.403114 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:03:34.198258 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:03:34.198219 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:03:42.211081 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:03:42.211043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:05:11.409968 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:11.409936 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:05:11.414783 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:11.414765 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:05:11.414922 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:11.414905 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:05:11.419828 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:11.419806 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:05:32.701150 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:32.701117 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:05:44.006999 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:44.006961 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:05:52.314781 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:05:52.314740 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:06:02.603071 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:06:02.602970 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:06:13.303941 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:06:13.303890 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:06:24.001302 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:06:24.001257 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:06:34.971604 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:06:34.971565 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:06:44.532445 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:06:44.532401 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:06:55.429195 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:06:55.429154 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:07:04.025680 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:07:04.025639 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:07:14.609812 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:07:14.609774 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:07:36.637124 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:07:36.637030 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:07:46.402278 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:07:46.402237 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:07:58.504257 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:07:58.504221 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:08:46.410198 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:08:46.410160 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:08:57.206976 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:08:57.206940 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:09:09.702837 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:09:09.702747 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:09:23.506971 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:09:23.506930 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:09:34.100045 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:09:34.099993 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:09:43.312593 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:09:43.312554 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:09:53.618037 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:09:53.617990 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:10:02.520825 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:02.520782 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:10:11.438412 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:11.438276 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:10:11.442406 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:11.440740 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:10:11.443030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:11.443010 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:10:11.445472 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:11.445453 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:10:12.007209 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:12.007174 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:10:25.012574 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:25.012537 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:10:34.344223 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:34.344132 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:10:44.614795 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:44.614757 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:10:55.417450 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:10:55.411811 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:11:07.016409 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:11:07.016368 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:11:17.406180 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:11:17.406140 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:11:29.516151 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:11:29.516108 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:11:41.702656 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:11:41.702615 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:11:52.707933 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:11:52.707889 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:12:05.113012 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:05.112913 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:12:17.006326 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:17.006274 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:12:23.425161 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.425127 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx"] Jul 20 17:12:23.428729 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.428710 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.431733 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.431710 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-82838e1d-openshift-default-dockercfg-9ff2t\"" Jul 20 17:12:23.461759 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.461726 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx"] Jul 20 17:12:23.588188 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588141 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-envoy\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588188 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588189 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-certs\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588470 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-data\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588470 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-credential-socket\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588470 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-socket\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588470 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588400 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-podinfo\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588470 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588445 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-token\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588470 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588468 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istiod-ca-cert\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.588658 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.588498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrfj9\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-kube-api-access-qrfj9\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689253 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-socket\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689253 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-podinfo\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689253 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689239 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-token\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689554 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istiod-ca-cert\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689554 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689537 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrfj9\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-kube-api-access-qrfj9\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689646 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689573 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-envoy\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689646 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-certs\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689768 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689611 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-socket\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689768 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689662 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-data\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.689874 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-credential-socket\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.690047 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.689973 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-certs\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.690123 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.690090 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-credential-socket\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.690209 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.690019 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-data\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.690209 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.690211 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istiod-ca-cert\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.691854 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.691826 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-podinfo\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.691972 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.691859 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-envoy\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.707495 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.707462 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-token\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.707660 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.707599 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrfj9\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-kube-api-access-qrfj9\") pod \"e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.739532 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.739490 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:23.925486 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.925456 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx"] Jul 20 17:12:23.926819 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:12:23.926796 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb6c4be_6b93_40fb_afb4_8bf041e188b1.slice/crio-782456676f2d7a5450eadb2adeaf8641618dc6f3cb6e618b577cbc2e12f0ccac WatchSource:0}: Error finding container 782456676f2d7a5450eadb2adeaf8641618dc6f3cb6e618b577cbc2e12f0ccac: Status 404 returned error can't find the container with id 782456676f2d7a5450eadb2adeaf8641618dc6f3cb6e618b577cbc2e12f0ccac Jul 20 17:12:23.928848 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.928818 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:12:23.928949 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.928900 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:12:23.929010 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:23.928948 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:12:24.822501 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:24.822461 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" event={"ID":"ddb6c4be-6b93-40fb-afb4-8bf041e188b1","Type":"ContainerStarted","Data":"486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1"} Jul 20 17:12:24.822902 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:24.822525 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" event={"ID":"ddb6c4be-6b93-40fb-afb4-8bf041e188b1","Type":"ContainerStarted","Data":"782456676f2d7a5450eadb2adeaf8641618dc6f3cb6e618b577cbc2e12f0ccac"} Jul 20 17:12:24.845753 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:24.845689 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podStartSLOduration=1.845670298 podStartE2EDuration="1.845670298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:12:24.844517154 +0000 UTC m=+1633.858939433" watchObservedRunningTime="2026-07-20 17:12:24.845670298 +0000 UTC m=+1633.860092533" Jul 20 17:12:25.740600 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:25.740556 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:25.745801 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:25.745770 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:25.827182 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:25.827149 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:25.828053 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:25.828036 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:33.007659 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:33.007617 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:12:40.930908 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:40.929442 2579 patch_prober.go:28] interesting pod/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": read tcp 10.132.0.2:54676->10.132.0.29:15021: read: connection reset by peer" start-of-body= Jul 20 17:12:40.930908 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:40.929567 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": read tcp 10.132.0.2:54676->10.132.0.29:15021: read: connection reset by peer" Jul 20 17:12:41.911517 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:41.911479 2579 generic.go:356] "Generic (PLEG): container finished" podID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerID="486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1" exitCode=0 Jul 20 17:12:41.911700 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:41.911557 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" event={"ID":"ddb6c4be-6b93-40fb-afb4-8bf041e188b1","Type":"ContainerDied","Data":"486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1"} Jul 20 17:12:41.912013 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:41.911995 2579 scope.go:117] "RemoveContainer" containerID="486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1" Jul 20 17:12:41.913140 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:41.913109 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:12:41.913211 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:41.913185 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:12:41.913260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:41.913216 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:12:42.917949 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:42.917919 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" event={"ID":"ddb6c4be-6b93-40fb-afb4-8bf041e188b1","Type":"ContainerStarted","Data":"c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45"} Jul 20 17:12:43.739902 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:43.739858 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:43.740113 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:43.740091 2579 patch_prober.go:28] interesting pod/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 20 17:12:43.740179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:43.740159 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 20 17:12:44.740655 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:44.740609 2579 patch_prober.go:28] interesting pod/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 20 17:12:44.741248 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:44.740681 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 20 17:12:45.740939 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:45.740893 2579 patch_prober.go:28] interesting pod/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 20 17:12:45.741380 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:45.740956 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 20 17:12:46.740165 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:46.740105 2579 patch_prober.go:28] interesting pod/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 20 17:12:46.740165 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:46.740160 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 20 17:12:47.743622 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:47.743591 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:47.744135 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:47.743884 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:12:47.744581 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:12:47.744565 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:13:05.211374 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:05.211337 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:13:07.086108 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:07.086072 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx"] Jul 20 17:13:07.086575 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:07.086335 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" containerID="cri-o://c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45" gracePeriod=30 Jul 20 17:13:12.350051 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.349982 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:13:12.447848 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.447818 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-socket\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448052 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.447872 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-certs\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448052 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.447890 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istiod-ca-cert\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448052 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.447925 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-data\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448052 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.447972 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-envoy\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448052 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.447988 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-podinfo\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448364 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448125 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-credential-socket\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448364 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448124 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:13:12.448364 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448172 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-token\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448364 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448214 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrfj9\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-kube-api-access-qrfj9\") pod \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\" (UID: \"ddb6c4be-6b93-40fb-afb4-8bf041e188b1\") " Jul 20 17:13:12.448364 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448171 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:13:12.448632 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448399 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:13:12.448632 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448433 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:13:12.448632 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448601 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.448632 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448617 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.448632 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448633 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.449036 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448645 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.449036 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.448902 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:13:12.450602 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.450574 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:13:12.450752 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.450691 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:13:12.450884 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.450867 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:13:12.450968 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.450948 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-kube-api-access-qrfj9" (OuterVolumeSpecName: "kube-api-access-qrfj9") pod "ddb6c4be-6b93-40fb-afb4-8bf041e188b1" (UID: "ddb6c4be-6b93-40fb-afb4-8bf041e188b1"). InnerVolumeSpecName "kube-api-access-qrfj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:13:12.549626 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.549584 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.549626 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.549616 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.549626 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.549626 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.549626 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.549640 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:12.549917 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:12.549650 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrfj9\" (UniqueName: \"kubernetes.io/projected/ddb6c4be-6b93-40fb-afb4-8bf041e188b1-kube-api-access-qrfj9\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:13:13.055701 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.055668 2579 generic.go:356] "Generic (PLEG): container finished" podID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerID="c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45" exitCode=0 Jul 20 17:13:13.055872 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.055714 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" event={"ID":"ddb6c4be-6b93-40fb-afb4-8bf041e188b1","Type":"ContainerDied","Data":"c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45"} Jul 20 17:13:13.055872 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.055757 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" event={"ID":"ddb6c4be-6b93-40fb-afb4-8bf041e188b1","Type":"ContainerDied","Data":"782456676f2d7a5450eadb2adeaf8641618dc6f3cb6e618b577cbc2e12f0ccac"} Jul 20 17:13:13.055872 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.055779 2579 scope.go:117] "RemoveContainer" containerID="c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45" Jul 20 17:13:13.055872 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.055778 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx" Jul 20 17:13:13.065607 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.065583 2579 scope.go:117] "RemoveContainer" containerID="486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1" Jul 20 17:13:13.074290 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.074269 2579 scope.go:117] "RemoveContainer" containerID="c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45" Jul 20 17:13:13.074629 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:13:13.074602 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45\": container with ID starting with c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45 not found: ID does not exist" containerID="c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45" Jul 20 17:13:13.074687 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.074643 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45"} err="failed to get container status \"c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45\": rpc error: code = NotFound desc = could not find container \"c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45\": container with ID starting with c826b17ce453403bb99ad5b9c881ab625ca860f52e20ca96aed770289e68fe45 not found: ID does not exist" Jul 20 17:13:13.074687 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.074665 2579 scope.go:117] "RemoveContainer" containerID="486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1" Jul 20 17:13:13.074908 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:13:13.074893 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1\": container with ID starting with 486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1 not found: ID does not exist" containerID="486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1" Jul 20 17:13:13.074949 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.074913 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1"} err="failed to get container status \"486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1\": rpc error: code = NotFound desc = could not find container \"486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1\": container with ID starting with 486051ca69aa6de21a4f161aa20aaf05ab5152c22e785156d9441fce99963ec1 not found: ID does not exist" Jul 20 17:13:13.081079 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.081043 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx"] Jul 20 17:13:13.085764 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.085732 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-82838e1d-openshift-default-75bf5d66f-8fqkx"] Jul 20 17:13:13.474728 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:13.474683 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" path="/var/lib/kubelet/pods/ddb6c4be-6b93-40fb-afb4-8bf041e188b1/volumes" Jul 20 17:13:17.024238 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:17.024202 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:13:53.302168 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:53.302132 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:13:55.531990 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.531952 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq"] Jul 20 17:13:55.532564 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.532423 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" Jul 20 17:13:55.532564 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.532443 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" Jul 20 17:13:55.532564 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.532461 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" Jul 20 17:13:55.532564 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.532469 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" Jul 20 17:13:55.532564 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.532564 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" Jul 20 17:13:55.532845 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.532576 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb6c4be-6b93-40fb-afb4-8bf041e188b1" containerName="istio-proxy" Jul 20 17:13:55.535035 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.535018 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538437 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538414 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e9002fce-openshift-default-dockercfg-bt8ln\"" Jul 20 17:13:55.538586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-certs\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538550 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-podinfo\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538695 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-envoy\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538695 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrv57\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-kube-api-access-jrv57\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538798 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-socket\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538852 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538802 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-credential-socket\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538852 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538842 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-data\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e32638-517e-4a96-9dd9-6c8bb112188e-istiod-ca-cert\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.538943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.538930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-token\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.560401 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.560355 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq"] Jul 20 17:13:55.639829 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-podinfo\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640020 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639835 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-envoy\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640020 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639861 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrv57\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-kube-api-access-jrv57\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640020 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639899 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-socket\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640020 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-credential-socket\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640020 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639938 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-data\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640020 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.639991 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e32638-517e-4a96-9dd9-6c8bb112188e-istiod-ca-cert\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640375 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.640046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-token\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640375 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.640088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-certs\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640375 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.640350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-socket\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640519 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.640504 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-certs\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640574 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.640541 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-data\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.640716 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.640622 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-credential-socket\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.641168 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.641148 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e32638-517e-4a96-9dd9-6c8bb112188e-istiod-ca-cert\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.642506 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.642482 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-envoy\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.643030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.643002 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-podinfo\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.649691 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.649660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrv57\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-kube-api-access-jrv57\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.655579 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.655542 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-token\") pod \"e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:55.845718 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:55.845607 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:56.006590 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.006407 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq"] Jul 20 17:13:56.009424 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:13:56.009380 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e32638_517e_4a96_9dd9_6c8bb112188e.slice/crio-af7e396e8f7c055732f2fada85c38307e22135c392301a53f1999072e8d86d5d WatchSource:0}: Error finding container af7e396e8f7c055732f2fada85c38307e22135c392301a53f1999072e8d86d5d: Status 404 returned error can't find the container with id af7e396e8f7c055732f2fada85c38307e22135c392301a53f1999072e8d86d5d Jul 20 17:13:56.011483 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.011450 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:56.011576 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.011538 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:56.011576 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.011567 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:13:56.254831 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.254797 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" event={"ID":"49e32638-517e-4a96-9dd9-6c8bb112188e","Type":"ContainerStarted","Data":"2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d"} Jul 20 17:13:56.255026 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.254858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" event={"ID":"49e32638-517e-4a96-9dd9-6c8bb112188e","Type":"ContainerStarted","Data":"af7e396e8f7c055732f2fada85c38307e22135c392301a53f1999072e8d86d5d"} Jul 20 17:13:56.282825 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.282767 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" podStartSLOduration=1.28274928 podStartE2EDuration="1.28274928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:13:56.281529477 +0000 UTC m=+1725.295951734" watchObservedRunningTime="2026-07-20 17:13:56.28274928 +0000 UTC m=+1725.297171610" Jul 20 17:13:56.846447 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:56.846403 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:57.851079 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:57.851038 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:58.264178 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:58.264144 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:13:58.265176 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:13:58.265152 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:04.121969 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:04.121929 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:14:12.338289 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:12.338203 2579 generic.go:356] "Generic (PLEG): container finished" podID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerID="2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d" exitCode=0 Jul 20 17:14:12.338289 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:12.338279 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" event={"ID":"49e32638-517e-4a96-9dd9-6c8bb112188e","Type":"ContainerDied","Data":"2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d"} Jul 20 17:14:12.338738 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:12.338715 2579 scope.go:117] "RemoveContainer" containerID="2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d" Jul 20 17:14:12.339756 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:12.339729 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:12.339830 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:12.339814 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:12.339878 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:12.339864 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:13.264907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:13.264844 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:13.345030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:13.344999 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" event={"ID":"49e32638-517e-4a96-9dd9-6c8bb112188e","Type":"ContainerStarted","Data":"87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963"} Jul 20 17:14:13.846368 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:13.846328 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:13.847585 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:13.847559 2579 patch_prober.go:28] interesting pod/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 20 17:14:13.847718 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:13.847619 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 20 17:14:14.846618 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:14.846574 2579 patch_prober.go:28] interesting pod/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 20 17:14:14.847096 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:14.846644 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 20 17:14:15.846980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:15.846941 2579 patch_prober.go:28] interesting pod/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 20 17:14:15.847417 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:15.846998 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 20 17:14:16.846727 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:16.846683 2579 patch_prober.go:28] interesting pod/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 20 17:14:16.846914 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:16.846748 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 20 17:14:17.850705 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:17.850678 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:17.851201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:17.851084 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:17.851819 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:17.851797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:36.909777 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:36.909731 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:14:39.794376 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:39.794342 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq"] Jul 20 17:14:39.794801 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:39.794578 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" containerID="cri-o://87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963" gracePeriod=30 Jul 20 17:14:40.580151 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.580109 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz"] Jul 20 17:14:40.583836 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.583815 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.589678 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.589653 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9185322f-openshift-default-dockercfg-t2cfr\"" Jul 20 17:14:40.600031 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.599990 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz"] Jul 20 17:14:40.767209 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767166 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-data\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767411 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-socket\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767411 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767279 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-envoy\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767411 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-credential-socket\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767411 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e786334-ce86-4ac9-923a-9b786834f543-istio-podinfo\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767583 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767460 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e786334-ce86-4ac9-923a-9b786834f543-istiod-ca-cert\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767583 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767498 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-certs\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767583 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767529 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sht7v\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-kube-api-access-sht7v\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.767708 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.767591 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-istio-token\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.868675 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e786334-ce86-4ac9-923a-9b786834f543-istiod-ca-cert\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.868675 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-certs\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.868675 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868651 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sht7v\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-kube-api-access-sht7v\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.868675 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-istio-token\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868707 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-data\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868761 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-socket\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-envoy\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-credential-socket\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.868972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e786334-ce86-4ac9-923a-9b786834f543-istio-podinfo\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.869115 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-certs\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869262 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.869141 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-data\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.869375 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e786334-ce86-4ac9-923a-9b786834f543-istiod-ca-cert\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.869441 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-credential-socket\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.869586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.869469 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-socket\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.871405 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.871384 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-envoy\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.871742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.871725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e786334-ce86-4ac9-923a-9b786834f543-istio-podinfo\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.877835 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.877810 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sht7v\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-kube-api-access-sht7v\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.877956 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.877906 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-istio-token\") pod \"e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:40.894079 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:40.894043 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:41.040148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.040110 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz"] Jul 20 17:14:41.043543 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:14:41.043514 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e786334_ce86_4ac9_923a_9b786834f543.slice/crio-b5cf9ad4a6c6a5c6e0d58043dd944ebd5b6a8fe4cb6ff22d2d26de8d7f132921 WatchSource:0}: Error finding container b5cf9ad4a6c6a5c6e0d58043dd944ebd5b6a8fe4cb6ff22d2d26de8d7f132921: Status 404 returned error can't find the container with id b5cf9ad4a6c6a5c6e0d58043dd944ebd5b6a8fe4cb6ff22d2d26de8d7f132921 Jul 20 17:14:41.046002 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.045974 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:41.046068 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.046042 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:41.046105 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.046068 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:41.481159 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.481108 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" event={"ID":"5e786334-ce86-4ac9-923a-9b786834f543","Type":"ContainerStarted","Data":"12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec"} Jul 20 17:14:41.481159 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.481167 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" event={"ID":"5e786334-ce86-4ac9-923a-9b786834f543","Type":"ContainerStarted","Data":"b5cf9ad4a6c6a5c6e0d58043dd944ebd5b6a8fe4cb6ff22d2d26de8d7f132921"} Jul 20 17:14:41.504933 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.504873 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" podStartSLOduration=1.504855401 podStartE2EDuration="1.504855401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:14:41.503338137 +0000 UTC m=+1770.517760388" watchObservedRunningTime="2026-07-20 17:14:41.504855401 +0000 UTC m=+1770.519277656" Jul 20 17:14:41.894360 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:41.894326 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:42.899028 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:42.898995 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:43.490371 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:43.490340 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:43.491542 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:43.491522 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:45.057066 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.057002 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:45.104562 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104529 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrv57\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-kube-api-access-jrv57\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104585 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-data\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104611 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-podinfo\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104625 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-socket\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104649 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-token\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104668 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-credential-socket\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104701 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-certs\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.104742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104734 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-envoy\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.105090 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104830 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e32638-517e-4a96-9dd9-6c8bb112188e-istiod-ca-cert\") pod \"49e32638-517e-4a96-9dd9-6c8bb112188e\" (UID: \"49e32638-517e-4a96-9dd9-6c8bb112188e\") " Jul 20 17:14:45.105090 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.104985 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:45.105090 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105007 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:45.105090 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105016 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:45.105299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105149 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.105299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105161 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.105299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105171 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.105550 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105422 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e32638-517e-4a96-9dd9-6c8bb112188e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:14:45.105832 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.105809 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-data" (OuterVolumeSpecName: "istio-data") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:45.106997 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.106974 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:14:45.107087 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.107011 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-kube-api-access-jrv57" (OuterVolumeSpecName: "kube-api-access-jrv57") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "kube-api-access-jrv57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:14:45.107516 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.107490 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-token" (OuterVolumeSpecName: "istio-token") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:14:45.107605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.107570 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49e32638-517e-4a96-9dd9-6c8bb112188e" (UID: "49e32638-517e-4a96-9dd9-6c8bb112188e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:14:45.206647 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.206608 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49e32638-517e-4a96-9dd9-6c8bb112188e-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.206647 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.206641 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrv57\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-kube-api-access-jrv57\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.206647 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.206651 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.206647 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.206660 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.206922 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.206669 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.206922 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.206677 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49e32638-517e-4a96-9dd9-6c8bb112188e-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:14:45.502754 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.502718 2579 generic.go:356] "Generic (PLEG): container finished" podID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerID="87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963" exitCode=0 Jul 20 17:14:45.502754 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.502757 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" Jul 20 17:14:45.503004 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.502787 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" event={"ID":"49e32638-517e-4a96-9dd9-6c8bb112188e","Type":"ContainerDied","Data":"87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963"} Jul 20 17:14:45.503004 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.502838 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq" event={"ID":"49e32638-517e-4a96-9dd9-6c8bb112188e","Type":"ContainerDied","Data":"af7e396e8f7c055732f2fada85c38307e22135c392301a53f1999072e8d86d5d"} Jul 20 17:14:45.503004 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.502864 2579 scope.go:117] "RemoveContainer" containerID="87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963" Jul 20 17:14:45.512731 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.512713 2579 scope.go:117] "RemoveContainer" containerID="2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d" Jul 20 17:14:45.521110 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.521091 2579 scope.go:117] "RemoveContainer" containerID="87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963" Jul 20 17:14:45.521430 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:14:45.521410 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963\": container with ID starting with 87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963 not found: ID does not exist" containerID="87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963" Jul 20 17:14:45.521487 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.521443 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963"} err="failed to get container status \"87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963\": rpc error: code = NotFound desc = could not find container \"87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963\": container with ID starting with 87bdb65c550ca58f3a51dedc6d127acd4ca7fea0ac47b7fb8ecb585e127c9963 not found: ID does not exist" Jul 20 17:14:45.521487 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.521463 2579 scope.go:117] "RemoveContainer" containerID="2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d" Jul 20 17:14:45.521721 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:14:45.521698 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d\": container with ID starting with 2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d not found: ID does not exist" containerID="2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d" Jul 20 17:14:45.521815 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.521728 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d"} err="failed to get container status \"2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d\": rpc error: code = NotFound desc = could not find container \"2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d\": container with ID starting with 2d6f569371ae74ba5bb0a260b1ec85aaa5b2cd444aabd186c6418b27685a3a0d not found: ID does not exist" Jul 20 17:14:45.526099 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.526071 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq"] Jul 20 17:14:45.530559 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:45.530536 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e9002fce-openshift-default-5486fdbcdc-5sscq"] Jul 20 17:14:47.475299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:47.475263 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" path="/var/lib/kubelet/pods/49e32638-517e-4a96-9dd9-6c8bb112188e/volumes" Jul 20 17:14:48.516237 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:48.516197 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:14:56.557987 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:56.557896 2579 generic.go:356] "Generic (PLEG): container finished" podID="5e786334-ce86-4ac9-923a-9b786834f543" containerID="12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec" exitCode=0 Jul 20 17:14:56.557987 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:56.557969 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" event={"ID":"5e786334-ce86-4ac9-923a-9b786834f543","Type":"ContainerDied","Data":"12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec"} Jul 20 17:14:56.558471 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:56.558391 2579 scope.go:117] "RemoveContainer" containerID="12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec" Jul 20 17:14:56.559460 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:56.559433 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:56.559532 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:56.559515 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:56.559586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:56.559569 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:14:57.564260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:57.564224 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" event={"ID":"5e786334-ce86-4ac9-923a-9b786834f543","Type":"ContainerStarted","Data":"42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874"} Jul 20 17:14:57.896463 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:57.896426 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:57.896921 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:57.896898 2579 patch_prober.go:28] interesting pod/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 20 17:14:57.897113 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:57.897088 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 20 17:14:58.490780 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:58.490744 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:14:58.895392 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:58.895348 2579 patch_prober.go:28] interesting pod/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 20 17:14:58.896022 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:58.895985 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 20 17:14:59.894575 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:59.894517 2579 patch_prober.go:28] interesting pod/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 20 17:14:59.894834 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:14:59.894805 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 20 17:15:00.895005 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:00.894960 2579 patch_prober.go:28] interesting pod/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 20 17:15:00.895659 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:00.895622 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 20 17:15:01.899606 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:01.899576 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:15:01.900775 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:01.900755 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:15:11.473176 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:11.473144 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:15:11.473745 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:11.473618 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:15:11.478494 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:11.478474 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:15:11.478681 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:11.478661 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:15:21.616176 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:21.616142 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:15:23.013979 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:23.013938 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz"] Jul 20 17:15:23.014451 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:23.014179 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" containerID="cri-o://42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874" gracePeriod=30 Jul 20 17:15:28.278235 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.278167 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:15:28.309368 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309339 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-credential-socket\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309368 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309371 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e786334-ce86-4ac9-923a-9b786834f543-istiod-ca-cert\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309394 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-socket\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309417 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-istio-token\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309471 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e786334-ce86-4ac9-923a-9b786834f543-istio-podinfo\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309495 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sht7v\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-kube-api-access-sht7v\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309529 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-certs\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309856 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309627 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-envoy\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309856 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309645 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:15:28.309856 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309694 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-data\") pod \"5e786334-ce86-4ac9-923a-9b786834f543\" (UID: \"5e786334-ce86-4ac9-923a-9b786834f543\") " Jul 20 17:15:28.309856 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309735 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:15:28.309856 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309834 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:15:28.309856 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.309852 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e786334-ce86-4ac9-923a-9b786834f543-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:15:28.310141 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.310023 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.310141 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.310044 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e786334-ce86-4ac9-923a-9b786834f543-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.310141 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.310058 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.310141 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.310071 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.310620 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.310600 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:15:28.311939 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.311913 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:15:28.312094 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.312054 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:15:28.312379 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.312361 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e786334-ce86-4ac9-923a-9b786834f543-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:15:28.312437 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.312372 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-kube-api-access-sht7v" (OuterVolumeSpecName: "kube-api-access-sht7v") pod "5e786334-ce86-4ac9-923a-9b786834f543" (UID: "5e786334-ce86-4ac9-923a-9b786834f543"). InnerVolumeSpecName "kube-api-access-sht7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:15:28.410949 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.410912 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e786334-ce86-4ac9-923a-9b786834f543-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.410949 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.410946 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sht7v\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-kube-api-access-sht7v\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.411181 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.410959 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.411181 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.410970 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e786334-ce86-4ac9-923a-9b786834f543-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.411181 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.410983 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e786334-ce86-4ac9-923a-9b786834f543-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:15:28.709446 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.709412 2579 generic.go:356] "Generic (PLEG): container finished" podID="5e786334-ce86-4ac9-923a-9b786834f543" containerID="42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874" exitCode=0 Jul 20 17:15:28.709620 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.709465 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" Jul 20 17:15:28.709620 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.709500 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" event={"ID":"5e786334-ce86-4ac9-923a-9b786834f543","Type":"ContainerDied","Data":"42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874"} Jul 20 17:15:28.709620 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.709553 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz" event={"ID":"5e786334-ce86-4ac9-923a-9b786834f543","Type":"ContainerDied","Data":"b5cf9ad4a6c6a5c6e0d58043dd944ebd5b6a8fe4cb6ff22d2d26de8d7f132921"} Jul 20 17:15:28.709620 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.709578 2579 scope.go:117] "RemoveContainer" containerID="42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874" Jul 20 17:15:28.719130 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.719111 2579 scope.go:117] "RemoveContainer" containerID="12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec" Jul 20 17:15:28.727659 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.727639 2579 scope.go:117] "RemoveContainer" containerID="42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874" Jul 20 17:15:28.727946 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:15:28.727927 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874\": container with ID starting with 42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874 not found: ID does not exist" containerID="42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874" Jul 20 17:15:28.728000 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.727955 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874"} err="failed to get container status \"42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874\": rpc error: code = NotFound desc = could not find container \"42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874\": container with ID starting with 42e14fb51a677009de42a31e0fa60db5d39392d60946fa670eeae4fe84fe1874 not found: ID does not exist" Jul 20 17:15:28.728000 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.727974 2579 scope.go:117] "RemoveContainer" containerID="12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec" Jul 20 17:15:28.728255 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:15:28.728236 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec\": container with ID starting with 12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec not found: ID does not exist" containerID="12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec" Jul 20 17:15:28.728339 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.728264 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec"} err="failed to get container status \"12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec\": rpc error: code = NotFound desc = could not find container \"12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec\": container with ID starting with 12bced6507a6dbfc585813012ae8ab6a809fccb68c43443947bba370cd9e2eec not found: ID does not exist" Jul 20 17:15:28.733779 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.733752 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz"] Jul 20 17:15:28.738781 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:28.738758 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9185322f-openshift-default-6bf89b8bc5-qwsvz"] Jul 20 17:15:29.474299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:15:29.474260 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e786334-ce86-4ac9-923a-9b786834f543" path="/var/lib/kubelet/pods/5e786334-ce86-4ac9-923a-9b786834f543/volumes" Jul 20 17:16:53.311759 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:16:53.311716 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:17:03.013317 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:03.013265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:17:08.380123 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380085 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64"] Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380439 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380451 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380466 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380472 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380478 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380484 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380491 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" Jul 20 17:17:08.380534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380496 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" Jul 20 17:17:08.380823 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380558 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" Jul 20 17:17:08.380823 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380566 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e32638-517e-4a96-9dd9-6c8bb112188e" containerName="istio-proxy" Jul 20 17:17:08.380823 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380572 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" Jul 20 17:17:08.380823 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.380580 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e786334-ce86-4ac9-923a-9b786834f543" containerName="istio-proxy" Jul 20 17:17:08.383908 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.383890 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.386953 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.386925 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fd39db2f-openshift-default-dockercfg-pmlt4\"" Jul 20 17:17:08.400692 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.400653 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64"] Jul 20 17:17:08.443966 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.443932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-certs\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.443966 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.443969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-podinfo\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444171 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444069 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn44x\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-kube-api-access-rn44x\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444171 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444102 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-socket\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444171 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-credential-socket\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444171 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-envoy\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444223 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-data\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-token\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.444299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.444293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istiod-ca-cert\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545672 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn44x\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-kube-api-access-rn44x\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545857 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-socket\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545857 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-credential-socket\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545857 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-envoy\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545857 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545741 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-data\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545857 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-token\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.545857 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545787 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istiod-ca-cert\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-certs\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.545885 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-podinfo\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.546118 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-socket\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.546222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-data\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.546247 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-credential-socket\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.546402 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-certs\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.546635 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.546558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istiod-ca-cert\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.548270 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.548249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-envoy\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.548500 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.548482 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-podinfo\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.555421 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.555386 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-token\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.555563 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.555486 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn44x\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-kube-api-access-rn44x\") pod \"e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.694761 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.694656 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:08.854965 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.854914 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64"] Jul 20 17:17:08.859465 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:17:08.859436 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726515d3_1ff5_48a4_9c4c_ceb5967e1007.slice/crio-13bae8fa117c4062606f6aba2b648142d02206e2f83edff557cbe9ba4171c4fa WatchSource:0}: Error finding container 13bae8fa117c4062606f6aba2b648142d02206e2f83edff557cbe9ba4171c4fa: Status 404 returned error can't find the container with id 13bae8fa117c4062606f6aba2b648142d02206e2f83edff557cbe9ba4171c4fa Jul 20 17:17:08.861486 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.861455 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:17:08.861600 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.861521 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:17:08.861600 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:08.861547 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:17:09.183499 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:09.183458 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" event={"ID":"726515d3-1ff5-48a4-9c4c-ceb5967e1007","Type":"ContainerStarted","Data":"61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4"} Jul 20 17:17:09.183688 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:09.183517 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" event={"ID":"726515d3-1ff5-48a4-9c4c-ceb5967e1007","Type":"ContainerStarted","Data":"13bae8fa117c4062606f6aba2b648142d02206e2f83edff557cbe9ba4171c4fa"} Jul 20 17:17:09.213735 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:09.213682 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" podStartSLOduration=1.213663142 podStartE2EDuration="1.213663142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:17:09.21148619 +0000 UTC m=+1918.225908465" watchObservedRunningTime="2026-07-20 17:17:09.213663142 +0000 UTC m=+1918.228085400" Jul 20 17:17:09.695260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:09.695215 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:09.700343 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:09.700293 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:10.187722 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:10.187695 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:10.188796 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:10.188773 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:15.309115 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:15.309075 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:17:23.248599 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:23.248565 2579 generic.go:356] "Generic (PLEG): container finished" podID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerID="61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4" exitCode=0 Jul 20 17:17:23.249025 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:23.248613 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" event={"ID":"726515d3-1ff5-48a4-9c4c-ceb5967e1007","Type":"ContainerDied","Data":"61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4"} Jul 20 17:17:23.249025 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:23.249016 2579 scope.go:117] "RemoveContainer" containerID="61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4" Jul 20 17:17:23.250091 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:23.250064 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:17:23.250145 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:23.250132 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:17:23.250181 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:23.250165 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:17:24.254673 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:24.254639 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" event={"ID":"726515d3-1ff5-48a4-9c4c-ceb5967e1007","Type":"ContainerStarted","Data":"5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc"} Jul 20 17:17:24.695471 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:24.695428 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:24.696481 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:24.696443 2579 patch_prober.go:28] interesting pod/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 20 17:17:24.696616 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:24.696523 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 20 17:17:25.188379 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:25.188331 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:25.696090 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:25.696044 2579 patch_prober.go:28] interesting pod/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 20 17:17:25.696648 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:25.696123 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 20 17:17:26.695641 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:26.695596 2579 patch_prober.go:28] interesting pod/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 20 17:17:26.695956 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:26.695923 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 20 17:17:27.695518 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:27.695475 2579 patch_prober.go:28] interesting pod/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 20 17:17:27.696156 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:27.696118 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 20 17:17:28.699059 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:28.699019 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:28.699969 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:28.699949 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:36.919609 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:36.919570 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:17:44.998523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:44.998486 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64"] Jul 20 17:17:44.999034 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:44.998758 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" containerID="cri-o://5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc" gracePeriod=30 Jul 20 17:17:50.252377 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.252295 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:50.345371 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345265 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-token\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345371 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345350 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-certs\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345587 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345375 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istiod-ca-cert\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345587 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345393 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn44x\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-kube-api-access-rn44x\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345587 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345415 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-credential-socket\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345587 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345448 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-socket\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345587 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345475 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-podinfo\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345587 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345513 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-data\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345591 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-envoy\") pod \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\" (UID: \"726515d3-1ff5-48a4-9c4c-ceb5967e1007\") " Jul 20 17:17:50.345890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345693 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:17:50.345890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345712 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:17:50.345890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345784 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:17:50.346102 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345919 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.346102 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345940 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.346102 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.345954 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.346345 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.346274 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:17:50.346854 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.346826 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-data" (OuterVolumeSpecName: "istio-data") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:17:50.347818 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.347794 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:17:50.347818 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.347810 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-token" (OuterVolumeSpecName: "istio-token") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:17:50.347935 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.347856 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:17:50.347935 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.347901 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-kube-api-access-rn44x" (OuterVolumeSpecName: "kube-api-access-rn44x") pod "726515d3-1ff5-48a4-9c4c-ceb5967e1007" (UID: "726515d3-1ff5-48a4-9c4c-ceb5967e1007"). InnerVolumeSpecName "kube-api-access-rn44x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:17:50.381325 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.381282 2579 generic.go:356] "Generic (PLEG): container finished" podID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerID="5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc" exitCode=0 Jul 20 17:17:50.381490 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.381351 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" Jul 20 17:17:50.381490 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.381393 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" event={"ID":"726515d3-1ff5-48a4-9c4c-ceb5967e1007","Type":"ContainerDied","Data":"5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc"} Jul 20 17:17:50.381490 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.381454 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64" event={"ID":"726515d3-1ff5-48a4-9c4c-ceb5967e1007","Type":"ContainerDied","Data":"13bae8fa117c4062606f6aba2b648142d02206e2f83edff557cbe9ba4171c4fa"} Jul 20 17:17:50.381490 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.381478 2579 scope.go:117] "RemoveContainer" containerID="5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc" Jul 20 17:17:50.390864 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.390844 2579 scope.go:117] "RemoveContainer" containerID="61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4" Jul 20 17:17:50.399227 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.399206 2579 scope.go:117] "RemoveContainer" containerID="5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc" Jul 20 17:17:50.399638 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:17:50.399619 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc\": container with ID starting with 5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc not found: ID does not exist" containerID="5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc" Jul 20 17:17:50.399724 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.399651 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc"} err="failed to get container status \"5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc\": rpc error: code = NotFound desc = could not find container \"5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc\": container with ID starting with 5582b4c75ee0f46d361bc5c204195091e47fe810d067b7110be4488990ad28bc not found: ID does not exist" Jul 20 17:17:50.399724 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.399677 2579 scope.go:117] "RemoveContainer" containerID="61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4" Jul 20 17:17:50.399945 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:17:50.399927 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4\": container with ID starting with 61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4 not found: ID does not exist" containerID="61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4" Jul 20 17:17:50.399997 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.399954 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4"} err="failed to get container status \"61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4\": rpc error: code = NotFound desc = could not find container \"61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4\": container with ID starting with 61982fbcc64fd10b9d3afd45e56e03208d25633a293b70d412b3a727985908c4 not found: ID does not exist" Jul 20 17:17:50.405947 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.405919 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64"] Jul 20 17:17:50.412034 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.412004 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fd39db2f-openshift-default-68bf9dfcff-lfq64"] Jul 20 17:17:50.446665 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.446627 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.446665 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.446660 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.446665 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.446671 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.446870 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.446679 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.446870 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.446687 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/726515d3-1ff5-48a4-9c4c-ceb5967e1007-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:50.446870 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:50.446695 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn44x\" (UniqueName: \"kubernetes.io/projected/726515d3-1ff5-48a4-9c4c-ceb5967e1007-kube-api-access-rn44x\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:17:51.474827 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:17:51.474792 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" path="/var/lib/kubelet/pods/726515d3-1ff5-48a4-9c4c-ceb5967e1007/volumes" Jul 20 17:18:04.751734 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.751653 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv"] Jul 20 17:18:04.752179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.751987 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" Jul 20 17:18:04.752179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.751998 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" Jul 20 17:18:04.752179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.752014 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" Jul 20 17:18:04.752179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.752020 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" Jul 20 17:18:04.752179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.752078 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" Jul 20 17:18:04.752179 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.752086 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="726515d3-1ff5-48a4-9c4c-ceb5967e1007" containerName="istio-proxy" Jul 20 17:18:04.755115 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.755092 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.758851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.758828 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fe047e55-openshift-default-dockercfg-fh86d\"" Jul 20 17:18:04.768054 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.768023 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv"] Jul 20 17:18:04.883108 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883074 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clk44\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-kube-api-access-clk44\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883108 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-certs\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883155 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-data\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883188 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-socket\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883212 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-token\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883249 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-credential-socket\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883267 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-envoy\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883284 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istiod-ca-cert\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.883402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.883388 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-podinfo\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984788 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984747 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-token\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-credential-socket\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-envoy\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984853 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istiod-ca-cert\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984878 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-podinfo\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984905 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clk44\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-kube-api-access-clk44\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.984980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-certs\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985263 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.984989 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-data\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985263 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.985042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-socket\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.985296 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-credential-socket\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985402 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.985373 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-data\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985499 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.985403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-certs\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985540 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.985511 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-socket\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.985956 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.985889 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istiod-ca-cert\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.987366 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.987344 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-envoy\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.987736 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.987718 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-podinfo\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.995131 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.995094 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-token\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:04.996648 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:04.996626 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clk44\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-kube-api-access-clk44\") pod \"e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:05.065375 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.065243 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:05.213611 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.213586 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv"] Jul 20 17:18:05.216514 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:18:05.216485 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff785267_c5bc_4c7d_a78a_f2cb0344ed0e.slice/crio-e6041ef0d5aa579f7b0fce8c0e2611296e3128a6463b75f45ecd9c0e346353f0 WatchSource:0}: Error finding container e6041ef0d5aa579f7b0fce8c0e2611296e3128a6463b75f45ecd9c0e346353f0: Status 404 returned error can't find the container with id e6041ef0d5aa579f7b0fce8c0e2611296e3128a6463b75f45ecd9c0e346353f0 Jul 20 17:18:05.218640 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.218608 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:18:05.218737 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.218674 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:18:05.218737 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.218701 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:18:05.451961 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.451917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" event={"ID":"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e","Type":"ContainerStarted","Data":"7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90"} Jul 20 17:18:05.452166 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.451980 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" event={"ID":"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e","Type":"ContainerStarted","Data":"e6041ef0d5aa579f7b0fce8c0e2611296e3128a6463b75f45ecd9c0e346353f0"} Jul 20 17:18:05.477103 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:05.477041 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" podStartSLOduration=1.477019675 podStartE2EDuration="1.477019675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:18:05.474006327 +0000 UTC m=+1974.488428584" watchObservedRunningTime="2026-07-20 17:18:05.477019675 +0000 UTC m=+1974.491441932" Jul 20 17:18:06.066349 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:06.066286 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:06.071218 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:06.071191 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:06.456728 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:06.456696 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:06.457669 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:06.457651 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:11.215374 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:11.215335 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:18:19.527801 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:19.527713 2579 generic.go:356] "Generic (PLEG): container finished" podID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerID="7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90" exitCode=0 Jul 20 17:18:19.527801 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:19.527772 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" event={"ID":"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e","Type":"ContainerDied","Data":"7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90"} Jul 20 17:18:19.528239 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:19.528130 2579 scope.go:117] "RemoveContainer" containerID="7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90" Jul 20 17:18:19.529214 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:19.529186 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:18:19.529289 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:19.529273 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:18:19.529358 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:19.529341 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:18:20.534579 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:20.534546 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" event={"ID":"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e","Type":"ContainerStarted","Data":"b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d"} Jul 20 17:18:21.066110 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:21.066064 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:21.067343 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:21.067295 2579 patch_prober.go:28] interesting pod/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 20 17:18:21.067421 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:21.067371 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 20 17:18:21.457257 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:21.457217 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:22.066835 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:22.066341 2579 patch_prober.go:28] interesting pod/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 20 17:18:22.066835 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:22.066418 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 20 17:18:22.504676 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:22.504611 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:18:23.066633 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:23.066583 2579 patch_prober.go:28] interesting pod/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 20 17:18:23.066862 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:23.066640 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 20 17:18:24.066201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:24.066160 2579 patch_prober.go:28] interesting pod/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 20 17:18:24.066417 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:24.066223 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 20 17:18:25.070458 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:25.070423 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:25.071523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:25.071504 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:18:42.206863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:42.206820 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:18:59.804024 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:18:59.803983 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:19:08.426894 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:19:08.426860 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:19:56.437360 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:19:56.437302 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv"] Jul 20 17:19:56.437932 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:19:56.437599 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" containerID="cri-o://b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d" gracePeriod=30 Jul 20 17:20:00.204281 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:00.204244 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:20:01.698398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.698369 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:20:01.759193 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759090 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-envoy\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759193 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759154 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-certs\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759193 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759178 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-token\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759536 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759212 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-credential-socket\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759536 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759244 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-socket\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759536 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759432 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istiod-ca-cert\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759536 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759472 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-podinfo\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759536 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759472 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:20:01.759536 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759520 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-data\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759828 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759545 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clk44\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-kube-api-access-clk44\") pod \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\" (UID: \"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e\") " Jul 20 17:20:01.759828 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759553 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:20:01.759828 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759569 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:20:01.759828 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759824 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.760030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759841 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.760030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759855 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.760030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.759854 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:20:01.760770 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.760736 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:20:01.761689 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.761649 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:20:01.761689 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.761683 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:20:01.761874 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.761727 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:20:01.762001 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.761986 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-kube-api-access-clk44" (OuterVolumeSpecName: "kube-api-access-clk44") pod "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" (UID: "ff785267-c5bc-4c7d-a78a-f2cb0344ed0e"). InnerVolumeSpecName "kube-api-access-clk44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:20:01.860666 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.860627 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.860666 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.860659 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clk44\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-kube-api-access-clk44\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.860666 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.860671 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.860666 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.860679 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.860945 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.860688 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:01.860945 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:01.860697 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:20:02.007034 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.006980 2579 generic.go:356] "Generic (PLEG): container finished" podID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerID="b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d" exitCode=0 Jul 20 17:20:02.007213 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.007041 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" Jul 20 17:20:02.007213 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.007064 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" event={"ID":"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e","Type":"ContainerDied","Data":"b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d"} Jul 20 17:20:02.007213 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.007119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv" event={"ID":"ff785267-c5bc-4c7d-a78a-f2cb0344ed0e","Type":"ContainerDied","Data":"e6041ef0d5aa579f7b0fce8c0e2611296e3128a6463b75f45ecd9c0e346353f0"} Jul 20 17:20:02.007213 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.007145 2579 scope.go:117] "RemoveContainer" containerID="b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d" Jul 20 17:20:02.016418 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.016400 2579 scope.go:117] "RemoveContainer" containerID="7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90" Jul 20 17:20:02.025408 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.025390 2579 scope.go:117] "RemoveContainer" containerID="b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d" Jul 20 17:20:02.025688 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:20:02.025668 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d\": container with ID starting with b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d not found: ID does not exist" containerID="b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d" Jul 20 17:20:02.025765 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.025701 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d"} err="failed to get container status \"b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d\": rpc error: code = NotFound desc = could not find container \"b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d\": container with ID starting with b5c2c59f0623606a77173f403921b2c2840b7411e9a952d865dedd484767263d not found: ID does not exist" Jul 20 17:20:02.025765 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.025726 2579 scope.go:117] "RemoveContainer" containerID="7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90" Jul 20 17:20:02.026007 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:20:02.025990 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90\": container with ID starting with 7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90 not found: ID does not exist" containerID="7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90" Jul 20 17:20:02.026050 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.026015 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90"} err="failed to get container status \"7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90\": rpc error: code = NotFound desc = could not find container \"7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90\": container with ID starting with 7adea9a6305304ee8afeb9b323d061c4c216699dce83d858cabd1123a24ada90 not found: ID does not exist" Jul 20 17:20:02.030078 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.030054 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv"] Jul 20 17:20:02.034032 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:02.034008 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fe047e55-openshift-default-d7656cbc5-ng9hv"] Jul 20 17:20:03.474834 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:03.474800 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" path="/var/lib/kubelet/pods/ff785267-c5bc-4c7d-a78a-f2cb0344ed0e/volumes" Jul 20 17:20:11.501419 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:11.501385 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:20:11.502617 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:11.502593 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:20:11.506259 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:11.506238 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:20:11.507385 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:11.507367 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:20:12.720005 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:12.719967 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:20:51.212279 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212236 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt"] Jul 20 17:20:51.212875 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212629 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" Jul 20 17:20:51.212875 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212643 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" Jul 20 17:20:51.212875 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212656 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" Jul 20 17:20:51.212875 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212662 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" Jul 20 17:20:51.212875 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212726 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" Jul 20 17:20:51.212875 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.212735 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff785267-c5bc-4c7d-a78a-f2cb0344ed0e" containerName="istio-proxy" Jul 20 17:20:51.214605 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.214585 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.217736 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.217718 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1d7b9a-openshift-default-dockercfg-lsrhz\"" Jul 20 17:20:51.228230 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.228196 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt"] Jul 20 17:20:51.315589 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315552 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-data\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.315589 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315593 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-socket\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.315814 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315612 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85cf319-0317-48e9-8345-8dd7f7991df4-istiod-ca-cert\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.315814 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-token\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.315814 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-462ck\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-kube-api-access-462ck\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.315814 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315758 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-credential-socket\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.315814 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-podinfo\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.316018 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-certs\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.316018 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.315846 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-envoy\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416699 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-data\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-socket\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85cf319-0317-48e9-8345-8dd7f7991df4-istiod-ca-cert\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-token\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416776 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-462ck\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-kube-api-access-462ck\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416826 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-credential-socket\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416871 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-podinfo\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.416907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416901 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-certs\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.417291 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.416922 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-envoy\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.417291 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.417258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-socket\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.417447 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.417257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-data\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.417447 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.417358 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-credential-socket\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.417558 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.417445 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-certs\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.417626 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.417604 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85cf319-0317-48e9-8345-8dd7f7991df4-istiod-ca-cert\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.419321 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.419286 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-envoy\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.419641 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.419623 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-podinfo\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.427108 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.427082 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-token\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.427446 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.427424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-462ck\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-kube-api-access-462ck\") pod \"e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.524754 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.524655 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:51.669650 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.669623 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt"] Jul 20 17:20:51.671620 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:20:51.671591 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85cf319_0317_48e9_8345_8dd7f7991df4.slice/crio-8e42223945cb4cc6d9cf789dd77f6389049feb8d3879a21acc97a493d544cce9 WatchSource:0}: Error finding container 8e42223945cb4cc6d9cf789dd77f6389049feb8d3879a21acc97a493d544cce9: Status 404 returned error can't find the container with id 8e42223945cb4cc6d9cf789dd77f6389049feb8d3879a21acc97a493d544cce9 Jul 20 17:20:51.673709 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.673677 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:20:51.673816 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.673743 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:20:51.673816 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:51.673771 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:20:52.235186 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:52.235140 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" event={"ID":"d85cf319-0317-48e9-8345-8dd7f7991df4","Type":"ContainerStarted","Data":"7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194"} Jul 20 17:20:52.235631 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:52.235243 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" event={"ID":"d85cf319-0317-48e9-8345-8dd7f7991df4","Type":"ContainerStarted","Data":"8e42223945cb4cc6d9cf789dd77f6389049feb8d3879a21acc97a493d544cce9"} Jul 20 17:20:52.260434 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:52.260380 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podStartSLOduration=1.2603636040000001 podStartE2EDuration="1.260363604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:20:52.258648923 +0000 UTC m=+2141.273071179" watchObservedRunningTime="2026-07-20 17:20:52.260363604 +0000 UTC m=+2141.274785859" Jul 20 17:20:52.525450 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:52.525341 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:52.530798 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:52.530769 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:53.240269 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:53.240225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:53.241283 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:53.241260 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:20:59.811390 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:20:59.811349 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:21:08.242614 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.242528 2579 patch_prober.go:28] interesting pod/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 20 17:21:08.242614 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.242585 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 20 17:21:08.320103 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.320016 2579 generic.go:356] "Generic (PLEG): container finished" podID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerID="7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194" exitCode=0 Jul 20 17:21:08.320103 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.320090 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" event={"ID":"d85cf319-0317-48e9-8345-8dd7f7991df4","Type":"ContainerDied","Data":"7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194"} Jul 20 17:21:08.320515 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.320498 2579 scope.go:117] "RemoveContainer" containerID="7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194" Jul 20 17:21:08.321495 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.321468 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:08.321594 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.321531 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:08.321594 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.321564 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:21:08.419808 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:08.419765 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:21:09.326236 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:09.326204 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" event={"ID":"d85cf319-0317-48e9-8345-8dd7f7991df4","Type":"ContainerStarted","Data":"0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc"} Jul 20 17:21:09.525734 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:09.525694 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:21:09.525960 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:09.525936 2579 patch_prober.go:28] interesting pod/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 20 17:21:09.526011 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:09.525988 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 20 17:21:10.525768 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:10.525723 2579 patch_prober.go:28] interesting pod/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 20 17:21:10.526170 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:10.525786 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 20 17:21:11.525634 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:11.525592 2579 patch_prober.go:28] interesting pod/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 20 17:21:11.525822 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:11.525669 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 20 17:21:12.525061 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:12.525016 2579 patch_prober.go:28] interesting pod/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 20 17:21:12.525278 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:12.525092 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 20 17:21:13.529458 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:13.529428 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:21:13.529847 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:13.529692 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:21:13.530338 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:13.530322 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:21:23.509926 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:23.509884 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:21:34.925341 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:21:34.925282 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:22:03.917529 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:22:03.917474 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:22:52.104501 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:22:52.104462 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:23:07.414032 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:07.413991 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:23:10.502244 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:10.502211 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt"] Jul 20 17:23:10.502679 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:10.502447 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" containerID="cri-o://0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc" gracePeriod=30 Jul 20 17:23:13.532681 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:13.532633 2579 patch_prober.go:28] interesting pod/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 17:23:13.533144 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:13.532700 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:23:15.758841 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.758807 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:23:15.803421 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803334 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-socket\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803421 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803390 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85cf319-0317-48e9-8345-8dd7f7991df4-istiod-ca-cert\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803645 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803429 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-462ck\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-kube-api-access-462ck\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803645 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803510 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-podinfo\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803645 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803550 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:15.803645 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803562 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-credential-socket\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803723 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-certs\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803752 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-envoy\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803773 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:15.803851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803794 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-token\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.803851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803792 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d85cf319-0317-48e9-8345-8dd7f7991df4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:23:15.803851 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803829 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-data\") pod \"d85cf319-0317-48e9-8345-8dd7f7991df4\" (UID: \"d85cf319-0317-48e9-8345-8dd7f7991df4\") " Jul 20 17:23:15.804139 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.803990 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:15.804196 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.804181 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.804256 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.804204 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85cf319-0317-48e9-8345-8dd7f7991df4-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.804256 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.804219 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.804256 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.804231 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.804817 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.804792 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-data" (OuterVolumeSpecName: "istio-data") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:15.805954 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.805929 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:23:15.806416 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.806395 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:23:15.806500 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.806426 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-kube-api-access-462ck" (OuterVolumeSpecName: "kube-api-access-462ck") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "kube-api-access-462ck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:23:15.806543 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.806505 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-token" (OuterVolumeSpecName: "istio-token") pod "d85cf319-0317-48e9-8345-8dd7f7991df4" (UID: "d85cf319-0317-48e9-8345-8dd7f7991df4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:23:15.905301 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.905245 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.905301 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.905293 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.905301 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.905337 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.905301 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.905346 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85cf319-0317-48e9-8345-8dd7f7991df4-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.905631 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.905355 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-462ck\" (UniqueName: \"kubernetes.io/projected/d85cf319-0317-48e9-8345-8dd7f7991df4-kube-api-access-462ck\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:23:15.913848 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.913813 2579 generic.go:356] "Generic (PLEG): container finished" podID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerID="0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc" exitCode=0 Jul 20 17:23:15.914030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.913868 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" Jul 20 17:23:15.914030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.913905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" event={"ID":"d85cf319-0317-48e9-8345-8dd7f7991df4","Type":"ContainerDied","Data":"0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc"} Jul 20 17:23:15.914030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.913959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt" event={"ID":"d85cf319-0317-48e9-8345-8dd7f7991df4","Type":"ContainerDied","Data":"8e42223945cb4cc6d9cf789dd77f6389049feb8d3879a21acc97a493d544cce9"} Jul 20 17:23:15.914030 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.913982 2579 scope.go:117] "RemoveContainer" containerID="0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc" Jul 20 17:23:15.923603 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.923584 2579 scope.go:117] "RemoveContainer" containerID="7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194" Jul 20 17:23:15.931968 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.931946 2579 scope.go:117] "RemoveContainer" containerID="0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc" Jul 20 17:23:15.932297 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:23:15.932279 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc\": container with ID starting with 0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc not found: ID does not exist" containerID="0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc" Jul 20 17:23:15.932396 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.932329 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc"} err="failed to get container status \"0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc\": rpc error: code = NotFound desc = could not find container \"0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc\": container with ID starting with 0bb3a268a9a144740a144d8a2dba335192f0dde37d7de41afda2725cd0642bcc not found: ID does not exist" Jul 20 17:23:15.932396 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.932347 2579 scope.go:117] "RemoveContainer" containerID="7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194" Jul 20 17:23:15.932624 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:23:15.932606 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194\": container with ID starting with 7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194 not found: ID does not exist" containerID="7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194" Jul 20 17:23:15.932674 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.932629 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194"} err="failed to get container status \"7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194\": rpc error: code = NotFound desc = could not find container \"7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194\": container with ID starting with 7111a92c34bb894cfbefe477c24b7252fb1c6853a2dcddca3475e54dbf800194 not found: ID does not exist" Jul 20 17:23:15.936730 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.936702 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt"] Jul 20 17:23:15.943577 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:15.943554 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1d7b9a-openshift-default-694db49fdf-b5vjt"] Jul 20 17:23:17.474540 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:17.474499 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" path="/var/lib/kubelet/pods/d85cf319-0317-48e9-8345-8dd7f7991df4/volumes" Jul 20 17:23:24.008472 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:24.008431 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:23:33.857013 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.856922 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g"] Jul 20 17:23:33.857426 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.857263 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" Jul 20 17:23:33.857426 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.857275 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" Jul 20 17:23:33.857426 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.857289 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" Jul 20 17:23:33.857426 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.857294 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" Jul 20 17:23:33.857426 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.857372 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" Jul 20 17:23:33.857426 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.857382 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d85cf319-0317-48e9-8345-8dd7f7991df4" containerName="istio-proxy" Jul 20 17:23:33.861764 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.861735 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.864858 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.864833 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-77a9c7-openshift-default-dockercfg-wll4j\"" Jul 20 17:23:33.874819 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.874790 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g"] Jul 20 17:23:33.972048 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-podinfo\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-socket\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972128 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-certs\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-credential-socket\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972260 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-token\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-envoy\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972283 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5fe4538-05fe-4e35-8042-c1bb5642c087-istiod-ca-cert\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972345 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f48nr\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-kube-api-access-f48nr\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:33.972520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:33.972374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-data\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073488 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-certs\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073488 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073486 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-credential-socket\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073524 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-token\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-envoy\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073566 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5fe4538-05fe-4e35-8042-c1bb5642c087-istiod-ca-cert\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f48nr\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-kube-api-access-f48nr\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-data\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-podinfo\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.073740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-socket\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.074103 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.073917 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-certs\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.074103 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.074088 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-socket\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.074192 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.074114 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-data\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.074238 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.074210 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-credential-socket\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.074596 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.074548 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5fe4538-05fe-4e35-8042-c1bb5642c087-istiod-ca-cert\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.076144 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.076122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-envoy\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.076486 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.076468 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-podinfo\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.082465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.082435 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-token\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.082683 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.082664 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f48nr\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-kube-api-access-f48nr\") pod \"e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.173734 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.173696 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:34.324880 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.324851 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g"] Jul 20 17:23:34.326680 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:23:34.326654 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5fe4538_05fe_4e35_8042_c1bb5642c087.slice/crio-994e6de1aefbf3f94b23fc6229c9dc1f3ea89b67139bcdcf17973a8286d6e9fd WatchSource:0}: Error finding container 994e6de1aefbf3f94b23fc6229c9dc1f3ea89b67139bcdcf17973a8286d6e9fd: Status 404 returned error can't find the container with id 994e6de1aefbf3f94b23fc6229c9dc1f3ea89b67139bcdcf17973a8286d6e9fd Jul 20 17:23:34.328792 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.328758 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:34.328890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.328830 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:34.328890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:34.328858 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:35.015214 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:35.015176 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" event={"ID":"e5fe4538-05fe-4e35-8042-c1bb5642c087","Type":"ContainerStarted","Data":"37a142e3603129cda6e357f69f51a4645af5329a9f3cf0a061ce5b32f7e14a2d"} Jul 20 17:23:35.015656 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:35.015235 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" event={"ID":"e5fe4538-05fe-4e35-8042-c1bb5642c087","Type":"ContainerStarted","Data":"994e6de1aefbf3f94b23fc6229c9dc1f3ea89b67139bcdcf17973a8286d6e9fd"} Jul 20 17:23:35.039243 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:35.039185 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podStartSLOduration=2.039164558 podStartE2EDuration="2.039164558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:23:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:23:35.037864509 +0000 UTC m=+2304.052286766" watchObservedRunningTime="2026-07-20 17:23:35.039164558 +0000 UTC m=+2304.053586813" Jul 20 17:23:35.173849 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:35.173806 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:35.178904 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:35.178873 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:36.020412 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:36.020382 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:36.021595 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:36.021577 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:42.223335 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.223266 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w"] Jul 20 17:23:42.233042 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.233012 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.236096 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.236070 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c63878-openshift-default-dockercfg-xbdgk\"" Jul 20 17:23:42.237593 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.237566 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w"] Jul 20 17:23:42.355119 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g5lx\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-kube-api-access-5g5lx\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355360 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-token\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355360 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-envoy\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355360 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355340 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-credential-socket\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-podinfo\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istiod-ca-cert\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355457 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-data\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355500 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-socket\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.355534 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.355521 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-certs\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456162 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456119 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-credential-socket\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456162 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456169 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-podinfo\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istiod-ca-cert\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456213 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-data\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456231 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-socket\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456246 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-certs\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g5lx\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-kube-api-access-5g5lx\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-token\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456465 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-envoy\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456824 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456706 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-socket\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456824 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456735 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-data\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456824 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-certs\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.456980 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.456960 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-credential-socket\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.457144 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.457125 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istiod-ca-cert\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.458865 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.458845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-envoy\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.459139 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.459119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-podinfo\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.465634 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.465603 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-token\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.465768 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.465745 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g5lx\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-kube-api-access-5g5lx\") pod \"e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.548337 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.548193 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:42.693985 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.693955 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w"] Jul 20 17:23:42.696700 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:23:42.696672 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d638a91_391d_4ec0_a103_64a3dd5f1c8d.slice/crio-7acb0230fc67186d013c65bb7f2655bebe5974f81beb3d7c24feca7b2a4540f9 WatchSource:0}: Error finding container 7acb0230fc67186d013c65bb7f2655bebe5974f81beb3d7c24feca7b2a4540f9: Status 404 returned error can't find the container with id 7acb0230fc67186d013c65bb7f2655bebe5974f81beb3d7c24feca7b2a4540f9 Jul 20 17:23:42.699287 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.699258 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:42.699365 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.699340 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:42.699411 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:42.699370 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:43.058214 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:43.058178 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" event={"ID":"6d638a91-391d-4ec0-a103-64a3dd5f1c8d","Type":"ContainerStarted","Data":"cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c"} Jul 20 17:23:43.058441 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:43.058232 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" event={"ID":"6d638a91-391d-4ec0-a103-64a3dd5f1c8d","Type":"ContainerStarted","Data":"7acb0230fc67186d013c65bb7f2655bebe5974f81beb3d7c24feca7b2a4540f9"} Jul 20 17:23:43.079954 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:43.079902 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podStartSLOduration=1.07988518 podStartE2EDuration="1.07988518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:23:43.079105524 +0000 UTC m=+2312.093527780" watchObservedRunningTime="2026-07-20 17:23:43.07988518 +0000 UTC m=+2312.094307436" Jul 20 17:23:43.548590 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:43.548547 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:43.553543 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:43.553518 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:44.065984 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:44.065953 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:44.067231 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:44.067204 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:23:44.312236 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:44.312200 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:23:52.061545 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:52.061485 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 17:23:52.062094 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:52.061560 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 17:23:52.590412 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:23:52.590374 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@7-27214-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5fe4538_05fe_4e35_8042_c1bb5642c087.slice/crio-conmon-37a142e3603129cda6e357f69f51a4645af5329a9f3cf0a061ce5b32f7e14a2d.scope\": RecentStats: unable to find data in memory cache]" Jul 20 17:23:53.120232 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:53.120196 2579 generic.go:356] "Generic (PLEG): container finished" podID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerID="37a142e3603129cda6e357f69f51a4645af5329a9f3cf0a061ce5b32f7e14a2d" exitCode=0 Jul 20 17:23:53.120653 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:53.120270 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" event={"ID":"e5fe4538-05fe-4e35-8042-c1bb5642c087","Type":"ContainerDied","Data":"37a142e3603129cda6e357f69f51a4645af5329a9f3cf0a061ce5b32f7e14a2d"} Jul 20 17:23:53.120722 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:53.120707 2579 scope.go:117] "RemoveContainer" containerID="37a142e3603129cda6e357f69f51a4645af5329a9f3cf0a061ce5b32f7e14a2d" Jul 20 17:23:53.121795 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:53.121762 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:53.121855 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:53.121841 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:53.121891 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:53.121883 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:23:54.131067 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:54.131017 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" event={"ID":"e5fe4538-05fe-4e35-8042-c1bb5642c087","Type":"ContainerStarted","Data":"25b170d2ef0451f702a19de0a1b93cf14e5b1241f942ef7e8533e76ce99e59a5"} Jul 20 17:23:54.174328 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:54.174250 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:54.174666 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:54.174643 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 20 17:23:54.174722 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:54.174693 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 20 17:23:55.175041 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:55.175004 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 20 17:23:55.175472 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:55.175066 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 20 17:23:56.174194 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:56.174156 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 20 17:23:56.174403 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:56.174214 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 20 17:23:57.174460 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:57.174422 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 20 17:23:57.174868 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:57.174511 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 20 17:23:58.175146 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:58.175087 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 20 17:23:58.175523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:58.175175 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 20 17:23:59.179424 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:59.179390 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:59.179901 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:59.179791 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:23:59.180735 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:23:59.180712 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:24:00.107533 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:00.107485 2579 patch_prober.go:28] interesting pod/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 17:24:00.107714 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:00.107552 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 17:24:00.125638 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:00.125603 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:24:01.172068 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:01.172035 2579 generic.go:356] "Generic (PLEG): container finished" podID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerID="cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c" exitCode=0 Jul 20 17:24:01.172554 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:01.172114 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" event={"ID":"6d638a91-391d-4ec0-a103-64a3dd5f1c8d","Type":"ContainerDied","Data":"cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c"} Jul 20 17:24:01.172603 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:01.172573 2579 scope.go:117] "RemoveContainer" containerID="cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c" Jul 20 17:24:01.173613 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:01.173589 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:24:01.173678 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:01.173654 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:24:01.173722 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:01.173685 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:24:02.181669 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:02.181634 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" event={"ID":"6d638a91-391d-4ec0-a103-64a3dd5f1c8d","Type":"ContainerStarted","Data":"3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270"} Jul 20 17:24:02.548648 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:02.548512 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:24:02.548796 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:02.548764 2579 patch_prober.go:28] interesting pod/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 20 17:24:02.548841 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:02.548823 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 20 17:24:03.549022 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:03.548980 2579 patch_prober.go:28] interesting pod/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 20 17:24:03.549571 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:03.549061 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 20 17:24:04.548576 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:04.548535 2579 patch_prober.go:28] interesting pod/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 20 17:24:04.548762 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:04.548593 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 20 17:24:05.549111 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:05.549068 2579 patch_prober.go:28] interesting pod/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 20 17:24:05.549563 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:05.549140 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 20 17:24:06.553001 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:06.552964 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:24:06.553411 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:06.553229 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:24:06.554133 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:06.554112 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:24:15.708221 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:15.708176 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:24:25.072618 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:25.072573 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g"] Jul 20 17:24:25.073089 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:25.072801 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" containerID="cri-o://25b170d2ef0451f702a19de0a1b93cf14e5b1241f942ef7e8533e76ce99e59a5" gracePeriod=30 Jul 20 17:24:29.182919 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:29.182882 2579 patch_prober.go:28] interesting pod/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 17:24:29.183361 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:29.182950 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:24:30.318506 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.318473 2579 generic.go:356] "Generic (PLEG): container finished" podID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerID="25b170d2ef0451f702a19de0a1b93cf14e5b1241f942ef7e8533e76ce99e59a5" exitCode=0 Jul 20 17:24:30.318935 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.318547 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" event={"ID":"e5fe4538-05fe-4e35-8042-c1bb5642c087","Type":"ContainerDied","Data":"25b170d2ef0451f702a19de0a1b93cf14e5b1241f942ef7e8533e76ce99e59a5"} Jul 20 17:24:30.318935 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.318635 2579 scope.go:117] "RemoveContainer" containerID="37a142e3603129cda6e357f69f51a4645af5329a9f3cf0a061ce5b32f7e14a2d" Jul 20 17:24:30.337196 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.337173 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:24:30.346925 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.346857 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-certs\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.346925 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.346901 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-podinfo\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.346925 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.346919 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-socket\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347135 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.346962 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5fe4538-05fe-4e35-8042-c1bb5642c087-istiod-ca-cert\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347135 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.346982 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f48nr\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-kube-api-access-f48nr\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347135 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347044 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-envoy\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347135 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347097 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-credential-socket\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347135 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347128 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-token\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347419 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347248 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:24:30.347419 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347335 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:24:30.347419 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347409 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5fe4538-05fe-4e35-8042-c1bb5642c087-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:24:30.347586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347425 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-data\") pod \"e5fe4538-05fe-4e35-8042-c1bb5642c087\" (UID: \"e5fe4538-05fe-4e35-8042-c1bb5642c087\") " Jul 20 17:24:30.347586 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347422 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:24:30.348013 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347807 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.348013 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347950 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.348013 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347977 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.348013 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.347990 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5fe4538-05fe-4e35-8042-c1bb5642c087-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.348645 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.348618 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:24:30.349567 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.349536 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:24:30.349667 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.349588 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-kube-api-access-f48nr" (OuterVolumeSpecName: "kube-api-access-f48nr") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "kube-api-access-f48nr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:24:30.349667 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.349589 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:24:30.349667 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.349624 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5fe4538-05fe-4e35-8042-c1bb5642c087" (UID: "e5fe4538-05fe-4e35-8042-c1bb5642c087"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:24:30.448523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.448482 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.448523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.448513 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.448523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.448521 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.448523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.448530 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5fe4538-05fe-4e35-8042-c1bb5642c087-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:30.448523 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:30.448539 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f48nr\" (UniqueName: \"kubernetes.io/projected/e5fe4538-05fe-4e35-8042-c1bb5642c087-kube-api-access-f48nr\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:24:31.325793 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:31.325761 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" Jul 20 17:24:31.325793 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:31.325774 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g" event={"ID":"e5fe4538-05fe-4e35-8042-c1bb5642c087","Type":"ContainerDied","Data":"994e6de1aefbf3f94b23fc6229c9dc1f3ea89b67139bcdcf17973a8286d6e9fd"} Jul 20 17:24:31.326293 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:31.325856 2579 scope.go:117] "RemoveContainer" containerID="25b170d2ef0451f702a19de0a1b93cf14e5b1241f942ef7e8533e76ce99e59a5" Jul 20 17:24:31.374688 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:31.374646 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g"] Jul 20 17:24:31.377612 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:31.377579 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-77a9c7-openshift-default-76fd468447-9lm7g"] Jul 20 17:24:31.475123 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:31.475082 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" path="/var/lib/kubelet/pods/e5fe4538-05fe-4e35-8042-c1bb5642c087/volumes" Jul 20 17:24:45.803753 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:24:45.803708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:25:01.896487 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:01.896447 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w"] Jul 20 17:25:01.897067 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:01.896708 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" containerID="cri-o://3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270" gracePeriod=30 Jul 20 17:25:06.556110 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:06.556065 2579 patch_prober.go:28] interesting pod/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 17:25:06.556608 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:06.556153 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 17:25:07.157969 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.157900 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:25:07.205733 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205696 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-data\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.205906 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205747 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-token\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.205906 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205792 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-credential-socket\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.205906 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205818 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-envoy\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.205906 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205866 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g5lx\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-kube-api-access-5g5lx\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.205906 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205894 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-podinfo\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.206128 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.205927 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-socket\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.206128 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206105 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-certs\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.206200 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206176 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istiod-ca-cert\") pod \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\" (UID: \"6d638a91-391d-4ec0-a103-64a3dd5f1c8d\") " Jul 20 17:25:07.206613 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206355 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:25:07.206613 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206581 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.206826 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206681 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:25:07.206894 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206818 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:25:07.206894 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.206852 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:25:07.207483 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.207076 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:25:07.208579 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.208556 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:25:07.208696 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.208637 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:25:07.208953 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.208917 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:25:07.209050 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.209022 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-kube-api-access-5g5lx" (OuterVolumeSpecName: "kube-api-access-5g5lx") pod "6d638a91-391d-4ec0-a103-64a3dd5f1c8d" (UID: "6d638a91-391d-4ec0-a103-64a3dd5f1c8d"). InnerVolumeSpecName "kube-api-access-5g5lx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:25:07.307504 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307454 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307504 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307497 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307504 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307509 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g5lx\" (UniqueName: \"kubernetes.io/projected/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-kube-api-access-5g5lx\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307504 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307520 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307790 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307529 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307790 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307537 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307790 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307546 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.307790 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.307554 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d638a91-391d-4ec0-a103-64a3dd5f1c8d-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:25:07.495323 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.495277 2579 generic.go:356] "Generic (PLEG): container finished" podID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerID="3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270" exitCode=0 Jul 20 17:25:07.495505 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.495351 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" Jul 20 17:25:07.495505 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.495361 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" event={"ID":"6d638a91-391d-4ec0-a103-64a3dd5f1c8d","Type":"ContainerDied","Data":"3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270"} Jul 20 17:25:07.495505 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.495420 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w" event={"ID":"6d638a91-391d-4ec0-a103-64a3dd5f1c8d","Type":"ContainerDied","Data":"7acb0230fc67186d013c65bb7f2655bebe5974f81beb3d7c24feca7b2a4540f9"} Jul 20 17:25:07.495505 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.495442 2579 scope.go:117] "RemoveContainer" containerID="3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270" Jul 20 17:25:07.504101 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.504078 2579 scope.go:117] "RemoveContainer" containerID="cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c" Jul 20 17:25:07.512578 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.512558 2579 scope.go:117] "RemoveContainer" containerID="3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270" Jul 20 17:25:07.512945 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:25:07.512922 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270\": container with ID starting with 3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270 not found: ID does not exist" containerID="3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270" Jul 20 17:25:07.512945 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.512951 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270"} err="failed to get container status \"3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270\": rpc error: code = NotFound desc = could not find container \"3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270\": container with ID starting with 3f1bf7437b2ea2167df1400d85b5d9e9d09285df4d28549e4f0b418161d78270 not found: ID does not exist" Jul 20 17:25:07.512945 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.512971 2579 scope.go:117] "RemoveContainer" containerID="cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c" Jul 20 17:25:07.513264 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:25:07.513242 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c\": container with ID starting with cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c not found: ID does not exist" containerID="cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c" Jul 20 17:25:07.513340 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.513270 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c"} err="failed to get container status \"cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c\": rpc error: code = NotFound desc = could not find container \"cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c\": container with ID starting with cd7e5b03d5d3743fbcd3b890eefa9ec314a768f525ce61254bf060e00482628c not found: ID does not exist" Jul 20 17:25:07.514203 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.514182 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w"] Jul 20 17:25:07.518109 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:07.518078 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c63878-openshift-default-694f4d5bff-hgz8w"] Jul 20 17:25:09.474802 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:09.474769 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" path="/var/lib/kubelet/pods/6d638a91-391d-4ec0-a103-64a3dd5f1c8d/volumes" Jul 20 17:25:11.531675 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:11.531646 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:25:11.533464 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:11.533440 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:25:11.539683 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:11.539658 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:25:11.541622 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:11.541603 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:25:22.771557 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771522 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847"] Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771849 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771860 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771880 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771886 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771894 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771899 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771910 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" Jul 20 17:25:22.771943 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771916 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" Jul 20 17:25:22.772186 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771966 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" Jul 20 17:25:22.772186 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771973 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" Jul 20 17:25:22.772186 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.771979 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5fe4538-05fe-4e35-8042-c1bb5642c087" containerName="istio-proxy" Jul 20 17:25:22.774046 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.774029 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.777026 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.777003 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f5995d-openshift-default-dockercfg-rqdkn\"" Jul 20 17:25:22.789398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.789367 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847"] Jul 20 17:25:22.850987 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.850940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b77gf\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-kube-api-access-b77gf\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851169 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-envoy\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851169 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851043 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-data\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851169 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-credential-socket\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851169 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdeb034-5234-434e-b083-88400bf3b49f-istio-podinfo\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851169 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851164 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-istio-token\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851358 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-certs\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851358 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851252 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdeb034-5234-434e-b083-88400bf3b49f-istiod-ca-cert\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.851358 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.851292 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-socket\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952380 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952337 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-credential-socket\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952563 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952389 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdeb034-5234-434e-b083-88400bf3b49f-istio-podinfo\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952563 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-istio-token\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952563 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-certs\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952737 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952622 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdeb034-5234-434e-b083-88400bf3b49f-istiod-ca-cert\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952737 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952672 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-socket\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952737 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b77gf\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-kube-api-access-b77gf\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952904 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952765 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-credential-socket\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952904 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952772 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-envoy\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.952904 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-data\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.953041 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952906 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-certs\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.953041 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.952974 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-socket\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.953221 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.953194 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-data\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.953328 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.953294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdeb034-5234-434e-b083-88400bf3b49f-istiod-ca-cert\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.955036 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.955011 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdeb034-5234-434e-b083-88400bf3b49f-istio-podinfo\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.955121 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.955069 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-envoy\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.961229 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.961203 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-istio-token\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:22.961388 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:22.961370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b77gf\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-kube-api-access-b77gf\") pod \"e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:23.084435 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.084337 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:23.227808 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.227772 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847"] Jul 20 17:25:23.229837 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:25:23.229806 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdeb034_5234_434e_b083_88400bf3b49f.slice/crio-6d7db9d769123bd5e7febc94c8d611113272386b580268574c50c45638a28f1d WatchSource:0}: Error finding container 6d7db9d769123bd5e7febc94c8d611113272386b580268574c50c45638a28f1d: Status 404 returned error can't find the container with id 6d7db9d769123bd5e7febc94c8d611113272386b580268574c50c45638a28f1d Jul 20 17:25:23.232138 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.232103 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:23.232254 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.232189 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:23.232254 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.232236 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:23.569110 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.569070 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" event={"ID":"fcdeb034-5234-434e-b083-88400bf3b49f","Type":"ContainerStarted","Data":"29f48953988daa949f033cc33eccdc34cba87304d09420d3e53bcfea9e552ef5"} Jul 20 17:25:23.569302 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.569136 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" event={"ID":"fcdeb034-5234-434e-b083-88400bf3b49f","Type":"ContainerStarted","Data":"6d7db9d769123bd5e7febc94c8d611113272386b580268574c50c45638a28f1d"} Jul 20 17:25:23.593041 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:23.592962 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" podStartSLOduration=1.592938582 podStartE2EDuration="1.592938582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:25:23.591371539 +0000 UTC m=+2412.605793797" watchObservedRunningTime="2026-07-20 17:25:23.592938582 +0000 UTC m=+2412.607360838" Jul 20 17:25:24.084866 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:24.084824 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:24.090102 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:24.090072 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:24.573951 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:24.573921 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:24.574890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:24.574873 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:38.449107 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:38.449064 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:25:46.685220 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:46.685128 2579 generic.go:356] "Generic (PLEG): container finished" podID="fcdeb034-5234-434e-b083-88400bf3b49f" containerID="29f48953988daa949f033cc33eccdc34cba87304d09420d3e53bcfea9e552ef5" exitCode=0 Jul 20 17:25:46.685220 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:46.685205 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" event={"ID":"fcdeb034-5234-434e-b083-88400bf3b49f","Type":"ContainerDied","Data":"29f48953988daa949f033cc33eccdc34cba87304d09420d3e53bcfea9e552ef5"} Jul 20 17:25:46.685767 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:46.685705 2579 scope.go:117] "RemoveContainer" containerID="29f48953988daa949f033cc33eccdc34cba87304d09420d3e53bcfea9e552ef5" Jul 20 17:25:46.686931 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:46.686902 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:46.687056 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:46.686978 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:46.687056 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:46.687011 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 20 17:25:47.691710 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:47.691668 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" event={"ID":"fcdeb034-5234-434e-b083-88400bf3b49f","Type":"ContainerStarted","Data":"aa214ba961c9cda2f9a98562c09bfea498a471b3ce1491dc85c747242d314441"} Jul 20 17:25:48.085569 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:48.085470 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:48.085752 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:48.085662 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 20 17:25:48.085752 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:48.085708 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 20 17:25:49.085874 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:49.085832 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 20 17:25:49.086335 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:49.085913 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 20 17:25:50.085059 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:50.085020 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 20 17:25:50.085244 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:50.085076 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 20 17:25:51.085119 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:51.085066 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 20 17:25:51.085567 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:51.085151 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 20 17:25:52.089080 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:52.089052 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:52.089576 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:52.089343 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:25:52.090143 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:25:52.090126 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:26:00.703730 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:26:00.703693 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:26:18.000742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:26:18.000703 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:27:04.728676 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.728587 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74"] Jul 20 17:27:04.729166 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.728979 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d638a91-391d-4ec0-a103-64a3dd5f1c8d" containerName="istio-proxy" Jul 20 17:27:04.730988 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.730963 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.735400 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.735372 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"auth-test-model-a24b97-kserve-self-signed-certs\"" Jul 20 17:27:04.735553 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.735398 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"kube-root-ca.crt\"" Jul 20 17:27:04.735553 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.735386 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"openshift-service-ca.crt\"" Jul 20 17:27:04.735553 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.735452 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"default-dockercfg-mmdtf\"" Jul 20 17:27:04.744253 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.744220 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74"] Jul 20 17:27:04.820240 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.820206 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-model-cache\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.820494 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.820247 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113a3efe-623f-4a85-88b3-dc30e09a144d-tls-certs\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.820494 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.820277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-dshm\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.820494 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.820445 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gggff\" (UniqueName: \"kubernetes.io/projected/113a3efe-623f-4a85-88b3-dc30e09a144d-kube-api-access-gggff\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.820720 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.820517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-tmp-dir\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.820720 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.820559 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-home\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.921821 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.921758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-home\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.922031 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.921840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-model-cache\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.922031 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.921876 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113a3efe-623f-4a85-88b3-dc30e09a144d-tls-certs\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.922031 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.921947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-dshm\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.922031 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.922030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gggff\" (UniqueName: \"kubernetes.io/projected/113a3efe-623f-4a85-88b3-dc30e09a144d-kube-api-access-gggff\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.922236 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.922071 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-tmp-dir\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.923598 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.922744 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-home\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.923807 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.923375 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-model-cache\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.923911 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.923381 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-tmp-dir\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.926863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.926804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113a3efe-623f-4a85-88b3-dc30e09a144d-tls-certs\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.927078 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.926398 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-dshm\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:04.932751 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:04.932710 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gggff\" (UniqueName: \"kubernetes.io/projected/113a3efe-623f-4a85-88b3-dc30e09a144d-kube-api-access-gggff\") pod \"auth-test-model-a24b97-kserve-74bb6bc595-mnv74\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:05.050117 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:05.050020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:05.199480 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:05.199443 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74"] Jul 20 17:27:05.202631 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:27:05.202588 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod113a3efe_623f_4a85_88b3_dc30e09a144d.slice/crio-1a25b286cd0a69fd18a8b2bdc6abfb28e39b6e9eb9fad9ccb8a6548e3927c0a5 WatchSource:0}: Error finding container 1a25b286cd0a69fd18a8b2bdc6abfb28e39b6e9eb9fad9ccb8a6548e3927c0a5: Status 404 returned error can't find the container with id 1a25b286cd0a69fd18a8b2bdc6abfb28e39b6e9eb9fad9ccb8a6548e3927c0a5 Jul 20 17:27:06.063509 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:06.063469 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" event={"ID":"113a3efe-623f-4a85-88b3-dc30e09a144d","Type":"ContainerStarted","Data":"57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1"} Jul 20 17:27:06.063509 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:06.063509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" event={"ID":"113a3efe-623f-4a85-88b3-dc30e09a144d","Type":"ContainerStarted","Data":"1a25b286cd0a69fd18a8b2bdc6abfb28e39b6e9eb9fad9ccb8a6548e3927c0a5"} Jul 20 17:27:06.081633 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:06.081576 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" podStartSLOduration=2.081557756 podStartE2EDuration="2.081557756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:27:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:27:06.080129611 +0000 UTC m=+2515.094551885" watchObservedRunningTime="2026-07-20 17:27:06.081557756 +0000 UTC m=+2515.095980029" Jul 20 17:27:08.307360 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:08.307323 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:27:15.051009 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:15.050948 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:15.051500 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:15.051026 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:15.063724 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:15.063696 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:15.114853 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:15.114820 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:27:30.266538 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:30.266492 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:27:49.510417 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:27:49.510374 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:28:03.608060 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:28:03.608019 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:28:19.806108 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:28:19.806064 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:28:39.606038 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:28:39.605957 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:28:57.803591 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:28:57.803552 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:29:13.059145 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:13.059101 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74"] Jul 20 17:29:14.602181 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:14.596631 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:29:25.602804 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:25.602762 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:29:28.080282 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.080223 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" podUID="113a3efe-623f-4a85-88b3-dc30e09a144d" containerName="main" containerID="cri-o://57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1" gracePeriod=45 Jul 20 17:29:28.336926 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.336856 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:29:28.366738 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.366706 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-model-cache\") pod \"113a3efe-623f-4a85-88b3-dc30e09a144d\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " Jul 20 17:29:28.366738 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.366743 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gggff\" (UniqueName: \"kubernetes.io/projected/113a3efe-623f-4a85-88b3-dc30e09a144d-kube-api-access-gggff\") pod \"113a3efe-623f-4a85-88b3-dc30e09a144d\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " Jul 20 17:29:28.366991 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.366767 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113a3efe-623f-4a85-88b3-dc30e09a144d-tls-certs\") pod \"113a3efe-623f-4a85-88b3-dc30e09a144d\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " Jul 20 17:29:28.366991 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.366787 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-tmp-dir\") pod \"113a3efe-623f-4a85-88b3-dc30e09a144d\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " Jul 20 17:29:28.366991 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.366811 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-home\") pod \"113a3efe-623f-4a85-88b3-dc30e09a144d\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " Jul 20 17:29:28.366991 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.366856 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-dshm\") pod \"113a3efe-623f-4a85-88b3-dc30e09a144d\" (UID: \"113a3efe-623f-4a85-88b3-dc30e09a144d\") " Jul 20 17:29:28.367205 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.367016 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-model-cache" (OuterVolumeSpecName: "model-cache") pod "113a3efe-623f-4a85-88b3-dc30e09a144d" (UID: "113a3efe-623f-4a85-88b3-dc30e09a144d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:29:28.367205 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.367184 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "113a3efe-623f-4a85-88b3-dc30e09a144d" (UID: "113a3efe-623f-4a85-88b3-dc30e09a144d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:29:28.367300 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.367201 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-home" (OuterVolumeSpecName: "home") pod "113a3efe-623f-4a85-88b3-dc30e09a144d" (UID: "113a3efe-623f-4a85-88b3-dc30e09a144d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:29:28.367572 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.367544 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-model-cache\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:29:28.367572 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.367570 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-tmp-dir\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:29:28.367715 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.367583 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-home\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:29:28.369181 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.369134 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113a3efe-623f-4a85-88b3-dc30e09a144d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "113a3efe-623f-4a85-88b3-dc30e09a144d" (UID: "113a3efe-623f-4a85-88b3-dc30e09a144d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 17:29:28.369279 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.369177 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113a3efe-623f-4a85-88b3-dc30e09a144d-kube-api-access-gggff" (OuterVolumeSpecName: "kube-api-access-gggff") pod "113a3efe-623f-4a85-88b3-dc30e09a144d" (UID: "113a3efe-623f-4a85-88b3-dc30e09a144d"). InnerVolumeSpecName "kube-api-access-gggff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:29:28.369436 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.369413 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-dshm" (OuterVolumeSpecName: "dshm") pod "113a3efe-623f-4a85-88b3-dc30e09a144d" (UID: "113a3efe-623f-4a85-88b3-dc30e09a144d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:29:28.468810 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.468773 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113a3efe-623f-4a85-88b3-dc30e09a144d-dshm\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:29:28.468810 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.468804 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gggff\" (UniqueName: \"kubernetes.io/projected/113a3efe-623f-4a85-88b3-dc30e09a144d-kube-api-access-gggff\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:29:28.468810 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.468815 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113a3efe-623f-4a85-88b3-dc30e09a144d-tls-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:29:28.729482 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.729447 2579 generic.go:356] "Generic (PLEG): container finished" podID="113a3efe-623f-4a85-88b3-dc30e09a144d" containerID="57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1" exitCode=0 Jul 20 17:29:28.729692 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.729495 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" Jul 20 17:29:28.729692 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.729523 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" event={"ID":"113a3efe-623f-4a85-88b3-dc30e09a144d","Type":"ContainerDied","Data":"57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1"} Jul 20 17:29:28.729692 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.729554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74" event={"ID":"113a3efe-623f-4a85-88b3-dc30e09a144d","Type":"ContainerDied","Data":"1a25b286cd0a69fd18a8b2bdc6abfb28e39b6e9eb9fad9ccb8a6548e3927c0a5"} Jul 20 17:29:28.729692 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.729572 2579 scope.go:117] "RemoveContainer" containerID="57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1" Jul 20 17:29:28.739594 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.739574 2579 scope.go:117] "RemoveContainer" containerID="57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1" Jul 20 17:29:28.739918 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:29:28.739899 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1\": container with ID starting with 57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1 not found: ID does not exist" containerID="57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1" Jul 20 17:29:28.739977 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.739929 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1"} err="failed to get container status \"57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1\": rpc error: code = NotFound desc = could not find container \"57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1\": container with ID starting with 57dbfbf703a71491a9ca19ea5c87ee648563f3d94a535a22479541f2fcc98fd1 not found: ID does not exist" Jul 20 17:29:28.751487 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.751453 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74"] Jul 20 17:29:28.755967 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:28.755938 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/auth-test-model-a24b97-kserve-74bb6bc595-mnv74"] Jul 20 17:29:29.474378 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:29.474342 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113a3efe-623f-4a85-88b3-dc30e09a144d" path="/var/lib/kubelet/pods/113a3efe-623f-4a85-88b3-dc30e09a144d/volumes" Jul 20 17:29:35.116091 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:35.116049 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:29:48.908234 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.908188 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc"] Jul 20 17:29:48.908739 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.908700 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113a3efe-623f-4a85-88b3-dc30e09a144d" containerName="main" Jul 20 17:29:48.908739 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.908720 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="113a3efe-623f-4a85-88b3-dc30e09a144d" containerName="main" Jul 20 17:29:48.908863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.908844 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="113a3efe-623f-4a85-88b3-dc30e09a144d" containerName="main" Jul 20 17:29:48.911848 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.911824 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:48.920914 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.920885 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"kube-root-ca.crt\"" Jul 20 17:29:48.920914 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.920905 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"sub-test-model-a24b97-kserve-self-signed-certs\"" Jul 20 17:29:48.923760 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.923740 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"default-dockercfg-mmdtf\"" Jul 20 17:29:48.923898 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.923880 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"openshift-service-ca.crt\"" Jul 20 17:29:48.945376 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:48.944744 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc"] Jul 20 17:29:49.061191 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.061151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52b1122-a3af-4fb3-9962-5e1397cd728d-tls-certs\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.061423 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.061243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lf4n\" (UniqueName: \"kubernetes.io/projected/d52b1122-a3af-4fb3-9962-5e1397cd728d-kube-api-access-2lf4n\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.061423 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.061273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-dshm\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.061423 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.061337 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-tmp-dir\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.061423 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.061368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-home\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.061597 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.061440 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-model-cache\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.162722 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.162621 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-model-cache\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.162722 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.162679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52b1122-a3af-4fb3-9962-5e1397cd728d-tls-certs\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.162988 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.162728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lf4n\" (UniqueName: \"kubernetes.io/projected/d52b1122-a3af-4fb3-9962-5e1397cd728d-kube-api-access-2lf4n\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.162988 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.162749 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-dshm\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.162988 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.162773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-tmp-dir\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.162988 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.162932 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-home\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.163201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.163152 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-model-cache\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.163201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.163178 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-tmp-dir\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.163276 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.163258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-home\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.165364 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.165336 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-dshm\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.165494 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.165476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52b1122-a3af-4fb3-9962-5e1397cd728d-tls-certs\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.171970 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.171948 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lf4n\" (UniqueName: \"kubernetes.io/projected/d52b1122-a3af-4fb3-9962-5e1397cd728d-kube-api-access-2lf4n\") pod \"sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.232787 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.232743 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:49.373526 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.373449 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc"] Jul 20 17:29:49.375941 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:29:49.375912 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd52b1122_a3af_4fb3_9962_5e1397cd728d.slice/crio-b183699ae68a6722638931ac4bfb17b7663ca7a233a2c28c7327ce9c982176ef WatchSource:0}: Error finding container b183699ae68a6722638931ac4bfb17b7663ca7a233a2c28c7327ce9c982176ef: Status 404 returned error can't find the container with id b183699ae68a6722638931ac4bfb17b7663ca7a233a2c28c7327ce9c982176ef Jul 20 17:29:49.835520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.835422 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" event={"ID":"d52b1122-a3af-4fb3-9962-5e1397cd728d","Type":"ContainerStarted","Data":"ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c"} Jul 20 17:29:49.835520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.835476 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" event={"ID":"d52b1122-a3af-4fb3-9962-5e1397cd728d","Type":"ContainerStarted","Data":"b183699ae68a6722638931ac4bfb17b7663ca7a233a2c28c7327ce9c982176ef"} Jul 20 17:29:49.854951 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:49.854896 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" podStartSLOduration=1.854870445 podStartE2EDuration="1.854870445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:29:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:29:49.853853222 +0000 UTC m=+2678.868275478" watchObservedRunningTime="2026-07-20 17:29:49.854870445 +0000 UTC m=+2678.869292700" Jul 20 17:29:52.499406 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:52.499365 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:29:59.233442 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:59.233400 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:59.233442 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:59.233453 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:59.246023 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:59.245996 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:29:59.898373 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:29:59.898336 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:30:11.563790 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:11.563712 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:30:11.566907 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:11.566883 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:30:11.568755 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:11.568738 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:30:11.571910 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:11.571887 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wmxlb_15a867d1-5903-4d89-96da-868c00db9f62/ovn-acl-logging/0.log" Jul 20 17:30:12.005112 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:12.005076 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:30:25.202688 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:25.202653 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc"] Jul 20 17:30:30.901515 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:30.901478 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:30:40.224012 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.223943 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" podUID="d52b1122-a3af-4fb3-9962-5e1397cd728d" containerName="main" containerID="cri-o://ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c" gracePeriod=45 Jul 20 17:30:40.489541 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.489513 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:30:40.641201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641163 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-dshm\") pod \"d52b1122-a3af-4fb3-9962-5e1397cd728d\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " Jul 20 17:30:40.641398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641216 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-model-cache\") pod \"d52b1122-a3af-4fb3-9962-5e1397cd728d\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " Jul 20 17:30:40.641398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641293 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-tmp-dir\") pod \"d52b1122-a3af-4fb3-9962-5e1397cd728d\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " Jul 20 17:30:40.641398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641344 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52b1122-a3af-4fb3-9962-5e1397cd728d-tls-certs\") pod \"d52b1122-a3af-4fb3-9962-5e1397cd728d\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " Jul 20 17:30:40.641398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641360 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lf4n\" (UniqueName: \"kubernetes.io/projected/d52b1122-a3af-4fb3-9962-5e1397cd728d-kube-api-access-2lf4n\") pod \"d52b1122-a3af-4fb3-9962-5e1397cd728d\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " Jul 20 17:30:40.641398 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641396 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-home\") pod \"d52b1122-a3af-4fb3-9962-5e1397cd728d\" (UID: \"d52b1122-a3af-4fb3-9962-5e1397cd728d\") " Jul 20 17:30:40.641701 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641590 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d52b1122-a3af-4fb3-9962-5e1397cd728d" (UID: "d52b1122-a3af-4fb3-9962-5e1397cd728d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:30:40.641701 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641626 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d52b1122-a3af-4fb3-9962-5e1397cd728d" (UID: "d52b1122-a3af-4fb3-9962-5e1397cd728d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:30:40.641808 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641727 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-home" (OuterVolumeSpecName: "home") pod "d52b1122-a3af-4fb3-9962-5e1397cd728d" (UID: "d52b1122-a3af-4fb3-9962-5e1397cd728d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:30:40.641859 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641840 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-home\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:30:40.641920 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641858 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-model-cache\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:30:40.641920 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.641874 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-tmp-dir\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:30:40.643602 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.643579 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-dshm" (OuterVolumeSpecName: "dshm") pod "d52b1122-a3af-4fb3-9962-5e1397cd728d" (UID: "d52b1122-a3af-4fb3-9962-5e1397cd728d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:30:40.644158 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.644142 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d52b1122-a3af-4fb3-9962-5e1397cd728d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d52b1122-a3af-4fb3-9962-5e1397cd728d" (UID: "d52b1122-a3af-4fb3-9962-5e1397cd728d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 17:30:40.644208 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.644184 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52b1122-a3af-4fb3-9962-5e1397cd728d-kube-api-access-2lf4n" (OuterVolumeSpecName: "kube-api-access-2lf4n") pod "d52b1122-a3af-4fb3-9962-5e1397cd728d" (UID: "d52b1122-a3af-4fb3-9962-5e1397cd728d"). InnerVolumeSpecName "kube-api-access-2lf4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:30:40.743206 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.743098 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d52b1122-a3af-4fb3-9962-5e1397cd728d-dshm\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:30:40.743206 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.743148 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d52b1122-a3af-4fb3-9962-5e1397cd728d-tls-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:30:40.743206 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.743161 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lf4n\" (UniqueName: \"kubernetes.io/projected/d52b1122-a3af-4fb3-9962-5e1397cd728d-kube-api-access-2lf4n\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:30:40.803113 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:40.803075 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:30:41.083244 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.083153 2579 generic.go:356] "Generic (PLEG): container finished" podID="d52b1122-a3af-4fb3-9962-5e1397cd728d" containerID="ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c" exitCode=0 Jul 20 17:30:41.083244 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.083201 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" Jul 20 17:30:41.083458 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.083243 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" event={"ID":"d52b1122-a3af-4fb3-9962-5e1397cd728d","Type":"ContainerDied","Data":"ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c"} Jul 20 17:30:41.083458 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.083287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc" event={"ID":"d52b1122-a3af-4fb3-9962-5e1397cd728d","Type":"ContainerDied","Data":"b183699ae68a6722638931ac4bfb17b7663ca7a233a2c28c7327ce9c982176ef"} Jul 20 17:30:41.083458 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.083338 2579 scope.go:117] "RemoveContainer" containerID="ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c" Jul 20 17:30:41.092550 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.092531 2579 scope.go:117] "RemoveContainer" containerID="ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c" Jul 20 17:30:41.092839 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:30:41.092820 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c\": container with ID starting with ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c not found: ID does not exist" containerID="ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c" Jul 20 17:30:41.092887 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.092849 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c"} err="failed to get container status \"ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c\": rpc error: code = NotFound desc = could not find container \"ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c\": container with ID starting with ed0e3bc95bb81e27fc11d806d4d984604d406a059222842434a1d6bcfdf7876c not found: ID does not exist" Jul 20 17:30:41.105657 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.105621 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc"] Jul 20 17:30:41.113001 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.112973 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/sub-test-model-a24b97-kserve-5c578dbcbf-dhmxc"] Jul 20 17:30:41.475762 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:30:41.475727 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d52b1122-a3af-4fb3-9962-5e1397cd728d" path="/var/lib/kubelet/pods/d52b1122-a3af-4fb3-9962-5e1397cd728d/volumes" Jul 20 17:31:13.020500 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.020463 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq"] Jul 20 17:31:13.020974 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.020809 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52b1122-a3af-4fb3-9962-5e1397cd728d" containerName="main" Jul 20 17:31:13.020974 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.020819 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52b1122-a3af-4fb3-9962-5e1397cd728d" containerName="main" Jul 20 17:31:13.020974 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.020880 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d52b1122-a3af-4fb3-9962-5e1397cd728d" containerName="main" Jul 20 17:31:13.024098 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.024079 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.031680 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.031648 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"kube-root-ca.crt\"" Jul 20 17:31:13.031825 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.031652 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"default-dockercfg-mmdtf\"" Jul 20 17:31:13.031825 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.031662 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"rate-test-model-a24b97-kserve-self-signed-certs\"" Jul 20 17:31:13.031825 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.031711 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a24b97\"/\"openshift-service-ca.crt\"" Jul 20 17:31:13.053067 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.053027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq"] Jul 20 17:31:13.143384 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.143338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-tmp-dir\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.143634 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.143446 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c18f095-0485-4950-91eb-8ca9624f4728-tls-certs\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.143634 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.143485 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-model-cache\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.143634 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.143522 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-dshm\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.143634 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.143605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-home\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.143808 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.143643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ds28\" (UniqueName: \"kubernetes.io/projected/0c18f095-0485-4950-91eb-8ca9624f4728-kube-api-access-5ds28\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.244538 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.244496 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c18f095-0485-4950-91eb-8ca9624f4728-tls-certs\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.244742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.244542 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-model-cache\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.244742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.244601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-dshm\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.244742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.244646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-home\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.244742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.244684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ds28\" (UniqueName: \"kubernetes.io/projected/0c18f095-0485-4950-91eb-8ca9624f4728-kube-api-access-5ds28\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.244742 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.244732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-tmp-dir\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.245083 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.245058 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-home\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.245201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.245089 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-model-cache\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.245201 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.245122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-tmp-dir\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.247101 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.247078 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-dshm\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.247268 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.247251 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c18f095-0485-4950-91eb-8ca9624f4728-tls-certs\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.255569 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.255543 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ds28\" (UniqueName: \"kubernetes.io/projected/0c18f095-0485-4950-91eb-8ca9624f4728-kube-api-access-5ds28\") pod \"rate-test-model-a24b97-kserve-7885b45fff-cjcvq\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.347524 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.347420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:13.486768 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:13.486736 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq"] Jul 20 17:31:13.490410 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:31:13.490381 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c18f095_0485_4950_91eb_8ca9624f4728.slice/crio-3428f450248134478b0ab79e30938839e6c5a3370143013209470e2569c796e5 WatchSource:0}: Error finding container 3428f450248134478b0ab79e30938839e6c5a3370143013209470e2569c796e5: Status 404 returned error can't find the container with id 3428f450248134478b0ab79e30938839e6c5a3370143013209470e2569c796e5 Jul 20 17:31:14.233343 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:14.233281 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" event={"ID":"0c18f095-0485-4950-91eb-8ca9624f4728","Type":"ContainerStarted","Data":"63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173"} Jul 20 17:31:14.233745 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:14.233356 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" event={"ID":"0c18f095-0485-4950-91eb-8ca9624f4728","Type":"ContainerStarted","Data":"3428f450248134478b0ab79e30938839e6c5a3370143013209470e2569c796e5"} Jul 20 17:31:14.260520 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:14.254381 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" podStartSLOduration=2.254358974 podStartE2EDuration="2.254358974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:31:14.25241869 +0000 UTC m=+2763.266840946" watchObservedRunningTime="2026-07-20 17:31:14.254358974 +0000 UTC m=+2763.268781231" Jul 20 17:31:16.807575 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:16.807527 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:31:23.348047 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:23.348007 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:23.348532 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:23.348065 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:23.360336 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:23.360283 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:24.292965 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:24.292936 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:31:31.806273 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:31.806235 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:31:48.503852 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:48.503761 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:31:57.913779 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:31:57.913749 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq"] Jul 20 17:32:03.010761 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:03.010716 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:32:12.934643 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:12.934569 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" podUID="0c18f095-0485-4950-91eb-8ca9624f4728" containerName="main" containerID="cri-o://63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173" gracePeriod=45 Jul 20 17:32:13.182474 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.182450 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:32:13.315007 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.314913 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ds28\" (UniqueName: \"kubernetes.io/projected/0c18f095-0485-4950-91eb-8ca9624f4728-kube-api-access-5ds28\") pod \"0c18f095-0485-4950-91eb-8ca9624f4728\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " Jul 20 17:32:13.315007 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.314965 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-tmp-dir\") pod \"0c18f095-0485-4950-91eb-8ca9624f4728\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " Jul 20 17:32:13.315007 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315000 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-model-cache\") pod \"0c18f095-0485-4950-91eb-8ca9624f4728\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " Jul 20 17:32:13.315341 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315029 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-home\") pod \"0c18f095-0485-4950-91eb-8ca9624f4728\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " Jul 20 17:32:13.315341 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315106 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c18f095-0485-4950-91eb-8ca9624f4728-tls-certs\") pod \"0c18f095-0485-4950-91eb-8ca9624f4728\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " Jul 20 17:32:13.315341 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315131 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-dshm\") pod \"0c18f095-0485-4950-91eb-8ca9624f4728\" (UID: \"0c18f095-0485-4950-91eb-8ca9624f4728\") " Jul 20 17:32:13.315341 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315259 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c18f095-0485-4950-91eb-8ca9624f4728" (UID: "0c18f095-0485-4950-91eb-8ca9624f4728"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:13.315568 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315293 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-home" (OuterVolumeSpecName: "home") pod "0c18f095-0485-4950-91eb-8ca9624f4728" (UID: "0c18f095-0485-4950-91eb-8ca9624f4728"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:13.315568 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315368 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c18f095-0485-4950-91eb-8ca9624f4728" (UID: "0c18f095-0485-4950-91eb-8ca9624f4728"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:13.315568 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315526 2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-tmp-dir\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.315568 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315546 2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-model-cache\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.315568 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.315558 2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-home\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.317299 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.317269 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c18f095-0485-4950-91eb-8ca9624f4728-kube-api-access-5ds28" (OuterVolumeSpecName: "kube-api-access-5ds28") pod "0c18f095-0485-4950-91eb-8ca9624f4728" (UID: "0c18f095-0485-4950-91eb-8ca9624f4728"). InnerVolumeSpecName "kube-api-access-5ds28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:13.317491 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.317471 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-dshm" (OuterVolumeSpecName: "dshm") pod "0c18f095-0485-4950-91eb-8ca9624f4728" (UID: "0c18f095-0485-4950-91eb-8ca9624f4728"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:13.317491 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.317475 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c18f095-0485-4950-91eb-8ca9624f4728-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c18f095-0485-4950-91eb-8ca9624f4728" (UID: "0c18f095-0485-4950-91eb-8ca9624f4728"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 17:32:13.416343 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.416286 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ds28\" (UniqueName: \"kubernetes.io/projected/0c18f095-0485-4950-91eb-8ca9624f4728-kube-api-access-5ds28\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.416343 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.416340 2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c18f095-0485-4950-91eb-8ca9624f4728-tls-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.416343 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.416350 2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c18f095-0485-4950-91eb-8ca9624f4728-dshm\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:13.504414 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.504381 2579 generic.go:356] "Generic (PLEG): container finished" podID="0c18f095-0485-4950-91eb-8ca9624f4728" containerID="63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173" exitCode=0 Jul 20 17:32:13.504567 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.504426 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" Jul 20 17:32:13.504567 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.504463 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" event={"ID":"0c18f095-0485-4950-91eb-8ca9624f4728","Type":"ContainerDied","Data":"63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173"} Jul 20 17:32:13.504567 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.504507 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq" event={"ID":"0c18f095-0485-4950-91eb-8ca9624f4728","Type":"ContainerDied","Data":"3428f450248134478b0ab79e30938839e6c5a3370143013209470e2569c796e5"} Jul 20 17:32:13.504567 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.504527 2579 scope.go:117] "RemoveContainer" containerID="63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173" Jul 20 17:32:13.514716 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.514692 2579 scope.go:117] "RemoveContainer" containerID="63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173" Jul 20 17:32:13.515120 ip-10-0-131-206 kubenswrapper[2579]: E0720 17:32:13.515092 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173\": container with ID starting with 63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173 not found: ID does not exist" containerID="63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173" Jul 20 17:32:13.515220 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.515127 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173"} err="failed to get container status \"63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173\": rpc error: code = NotFound desc = could not find container \"63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173\": container with ID starting with 63f9c441e4c3fa6f72f84addd2f89b25f19ae9f180b89619ca341373e74b2173 not found: ID does not exist" Jul 20 17:32:13.525136 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.525105 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq"] Jul 20 17:32:13.527251 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:13.527217 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a24b97/rate-test-model-a24b97-kserve-7885b45fff-cjcvq"] Jul 20 17:32:15.475295 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:15.475249 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c18f095-0485-4950-91eb-8ca9624f4728" path="/var/lib/kubelet/pods/0c18f095-0485-4950-91eb-8ca9624f4728/volumes" Jul 20 17:32:30.399633 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:30.399590 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:32:38.484595 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:38.484556 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847"] Jul 20 17:32:38.485193 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:38.484810 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" containerID="cri-o://aa214ba961c9cda2f9a98562c09bfea498a471b3ce1491dc85c747242d314441" gracePeriod=30 Jul 20 17:32:43.651820 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.651784 2579 generic.go:356] "Generic (PLEG): container finished" podID="fcdeb034-5234-434e-b083-88400bf3b49f" containerID="aa214ba961c9cda2f9a98562c09bfea498a471b3ce1491dc85c747242d314441" exitCode=0 Jul 20 17:32:43.652336 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.651868 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" event={"ID":"fcdeb034-5234-434e-b083-88400bf3b49f","Type":"ContainerDied","Data":"aa214ba961c9cda2f9a98562c09bfea498a471b3ce1491dc85c747242d314441"} Jul 20 17:32:43.652336 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.651942 2579 scope.go:117] "RemoveContainer" containerID="29f48953988daa949f033cc33eccdc34cba87304d09420d3e53bcfea9e552ef5" Jul 20 17:32:43.752142 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.752072 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:32:43.899956 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.899923 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdeb034-5234-434e-b083-88400bf3b49f-istio-podinfo\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.899970 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b77gf\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-kube-api-access-b77gf\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.899985 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-istio-token\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900024 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-envoy\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900043 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-data\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900090 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-certs\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900110 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdeb034-5234-434e-b083-88400bf3b49f-istiod-ca-cert\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900148 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900131 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-credential-socket\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900549 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900178 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-socket\") pod \"fcdeb034-5234-434e-b083-88400bf3b49f\" (UID: \"fcdeb034-5234-434e-b083-88400bf3b49f\") " Jul 20 17:32:43.900549 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900505 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:43.900667 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900606 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:43.900725 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900700 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:43.900834 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.900772 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcdeb034-5234-434e-b083-88400bf3b49f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 17:32:43.901253 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.901232 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:43.903122 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.903078 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-kube-api-access-b77gf" (OuterVolumeSpecName: "kube-api-access-b77gf") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "kube-api-access-b77gf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:43.903122 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.903093 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 17:32:43.903284 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.903139 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcdeb034-5234-434e-b083-88400bf3b49f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 17:32:43.903284 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:43.903143 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcdeb034-5234-434e-b083-88400bf3b49f" (UID: "fcdeb034-5234-434e-b083-88400bf3b49f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 17:32:44.001618 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001581 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-envoy\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001618 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001613 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-istio-data\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001618 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001626 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-certs\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001634 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcdeb034-5234-434e-b083-88400bf3b49f-istiod-ca-cert\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001642 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-credential-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001651 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcdeb034-5234-434e-b083-88400bf3b49f-workload-socket\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001661 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcdeb034-5234-434e-b083-88400bf3b49f-istio-podinfo\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001669 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b77gf\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-kube-api-access-b77gf\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.001863 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.001677 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcdeb034-5234-434e-b083-88400bf3b49f-istio-token\") on node \"ip-10-0-131-206.ec2.internal\" DevicePath \"\"" Jul 20 17:32:44.657924 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.657894 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" Jul 20 17:32:44.658386 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.657896 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847" event={"ID":"fcdeb034-5234-434e-b083-88400bf3b49f","Type":"ContainerDied","Data":"6d7db9d769123bd5e7febc94c8d611113272386b580268574c50c45638a28f1d"} Jul 20 17:32:44.658386 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.658031 2579 scope.go:117] "RemoveContainer" containerID="aa214ba961c9cda2f9a98562c09bfea498a471b3ce1491dc85c747242d314441" Jul 20 17:32:44.681065 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.681033 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847"] Jul 20 17:32:44.686893 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:44.686858 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f5995d-openshift-default-7dbd75f75-qp847"] Jul 20 17:32:45.474839 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:45.474799 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" path="/var/lib/kubelet/pods/fcdeb034-5234-434e-b083-88400bf3b49f/volumes" Jul 20 17:32:52.496047 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:32:52.496009 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7fkl7"] Jul 20 17:33:07.254515 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:07.254439 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-7jsnt_eeb0e651-9ed6-4579-bb42-58b549be40d1/copy-manifests/0.log" Jul 20 17:33:07.262750 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:07.262725 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-7jsnt_eeb0e651-9ed6-4579-bb42-58b549be40d1/manager/0.log" Jul 20 17:33:10.362415 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:10.362385 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7fkl7_c590f4c9-16b8-42b2-acf3-709f7813e6ee/limitador/0.log" Jul 20 17:33:10.836607 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:10.836571 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zzb74_29a0099a-00f5-4c62-ad2f-b340fd793176/istio-proxy/0.log" Jul 20 17:33:10.963676 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:10.963640 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-brv2p_cfea2583-c0b7-4473-933c-02d489624373/discovery/0.log" Jul 20 17:33:11.324999 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:11.324968 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/1.log" Jul 20 17:33:11.684435 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:11.684409 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d855b796b-mp8tb_2d29f98c-cfe7-4be3-b9f7-3980bc6f718e/router/0.log" Jul 20 17:33:12.386689 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:12.386662 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9qdgq_78e9972a-4471-4d58-b602-4c25f727628d/main/0.log" Jul 20 17:33:20.875538 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:20.875505 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4lj79_416d3c60-3d18-4f9e-a38f-4e5b085467d9/global-pull-secret-syncer/0.log" Jul 20 17:33:21.096727 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:21.096691 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zzjt5_e8e062be-3e22-4169-ae03-1877d52568ff/konnectivity-agent/0.log" Jul 20 17:33:21.172621 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:21.172585 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-206.ec2.internal_53461b03e56e0494bd965b8a7e0d36e4/haproxy/0.log" Jul 20 17:33:26.786075 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:26.786041 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7fkl7_c590f4c9-16b8-42b2-acf3-709f7813e6ee/limitador/0.log" Jul 20 17:33:28.730163 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:28.730122 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-56dlt_22e6b30a-676c-4f8a-85e4-620986107327/cluster-monitoring-operator/0.log" Jul 20 17:33:28.934091 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:28.934062 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8zc4l_5cb6683c-72ce-47b7-a905-552a3d6133d3/node-exporter/0.log" Jul 20 17:33:28.955890 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:28.955864 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8zc4l_5cb6683c-72ce-47b7-a905-552a3d6133d3/kube-rbac-proxy/0.log" Jul 20 17:33:28.977850 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:28.977826 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8zc4l_5cb6683c-72ce-47b7-a905-552a3d6133d3/init-textfile/0.log" Jul 20 17:33:29.267324 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.267222 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/prometheus/0.log" Jul 20 17:33:29.291938 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.291902 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/config-reloader/0.log" Jul 20 17:33:29.316057 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.316025 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/thanos-sidecar/0.log" Jul 20 17:33:29.338184 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.338152 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/kube-rbac-proxy-web/0.log" Jul 20 17:33:29.365240 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.365210 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/kube-rbac-proxy/0.log" Jul 20 17:33:29.390721 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.390691 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/kube-rbac-proxy-thanos/0.log" Jul 20 17:33:29.415688 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:29.415665 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_438f4edb-a383-4879-b8dd-7173c6e41e80/init-config-reloader/0.log" Jul 20 17:33:31.482458 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.482428 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/1.log" Jul 20 17:33:31.487521 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.487501 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ztj6t_cef8a584-a9c8-4df8-80e9-f61b935eb73d/console-operator/2.log" Jul 20 17:33:31.533997 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.533962 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z"] Jul 20 17:33:31.534376 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534359 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c18f095-0485-4950-91eb-8ca9624f4728" containerName="main" Jul 20 17:33:31.534425 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534379 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c18f095-0485-4950-91eb-8ca9624f4728" containerName="main" Jul 20 17:33:31.534425 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534390 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" Jul 20 17:33:31.534425 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534396 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" Jul 20 17:33:31.534524 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534458 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" Jul 20 17:33:31.534524 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534468 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c18f095-0485-4950-91eb-8ca9624f4728" containerName="main" Jul 20 17:33:31.534524 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.534475 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdeb034-5234-434e-b083-88400bf3b49f" containerName="istio-proxy" Jul 20 17:33:31.537663 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.537646 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.540733 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.540705 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z68mp\"/\"openshift-service-ca.crt\"" Jul 20 17:33:31.540911 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.540756 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z68mp\"/\"kube-root-ca.crt\"" Jul 20 17:33:31.542270 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.542254 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z68mp\"/\"default-dockercfg-58xlt\"" Jul 20 17:33:31.547047 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.547020 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z"] Jul 20 17:33:31.629514 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.629475 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-sys\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.629514 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.629516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-podres\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.629740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.629535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sztbm\" (UniqueName: \"kubernetes.io/projected/2a5a3629-7751-4a70-8888-5e6e63af61d1-kube-api-access-sztbm\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.629740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.629627 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-lib-modules\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.629740 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.629641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-proc\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731136 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-lib-modules\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731136 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-proc\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731432 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-sys\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731432 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731208 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-podres\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731432 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sztbm\" (UniqueName: \"kubernetes.io/projected/2a5a3629-7751-4a70-8888-5e6e63af61d1-kube-api-access-sztbm\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731432 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731300 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-lib-modules\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731432 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731347 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-proc\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731432 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731299 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-sys\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.731685 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.731439 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2a5a3629-7751-4a70-8888-5e6e63af61d1-podres\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.739871 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.739811 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sztbm\" (UniqueName: \"kubernetes.io/projected/2a5a3629-7751-4a70-8888-5e6e63af61d1-kube-api-access-sztbm\") pod \"perf-node-gather-daemonset-8rr6z\" (UID: \"2a5a3629-7751-4a70-8888-5e6e63af61d1\") " pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.849354 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.849302 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:31.991606 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:31.991575 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z"] Jul 20 17:33:31.994598 ip-10-0-131-206 kubenswrapper[2579]: W0720 17:33:31.994570 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2a5a3629_7751_4a70_8888_5e6e63af61d1.slice/crio-6b41baa2019d3859ea0ab11ecac891f8839dc7e6b1fbcc0abe401aed422d8d87 WatchSource:0}: Error finding container 6b41baa2019d3859ea0ab11ecac891f8839dc7e6b1fbcc0abe401aed422d8d87: Status 404 returned error can't find the container with id 6b41baa2019d3859ea0ab11ecac891f8839dc7e6b1fbcc0abe401aed422d8d87 Jul 20 17:33:32.137109 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:32.137081 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-xjm4c_b0203aa3-6bb1-44ea-9b29-cfd518891282/download-server/0.log" Jul 20 17:33:32.809524 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:32.809491 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-xmj6t_19d59c2a-8667-4d18-a446-dc902d612c5f/volume-data-source-validator/0.log" Jul 20 17:33:32.875120 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:32.875085 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" event={"ID":"2a5a3629-7751-4a70-8888-5e6e63af61d1","Type":"ContainerStarted","Data":"13990acbf4924d9196bd9e64392c094cc7ab8560dd27939142ffbba56000eed6"} Jul 20 17:33:32.875297 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:32.875169 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" event={"ID":"2a5a3629-7751-4a70-8888-5e6e63af61d1","Type":"ContainerStarted","Data":"6b41baa2019d3859ea0ab11ecac891f8839dc7e6b1fbcc0abe401aed422d8d87"} Jul 20 17:33:32.875297 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:32.875230 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:32.894055 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:32.894000 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" podStartSLOduration=1.893984294 podStartE2EDuration="1.893984294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:33:32.891375657 +0000 UTC m=+2901.905797916" watchObservedRunningTime="2026-07-20 17:33:32.893984294 +0000 UTC m=+2901.908406550" Jul 20 17:33:33.866270 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:33.866241 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kc8wg_1f986628-3da0-4d06-853e-634332ca7563/dns/0.log" Jul 20 17:33:33.887408 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:33.887381 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kc8wg_1f986628-3da0-4d06-853e-634332ca7563/kube-rbac-proxy/0.log" Jul 20 17:33:34.005093 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:34.005060 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krvrr_a6a434ee-9394-44d3-bfa8-de9ee981cc63/dns-node-resolver/0.log" Jul 20 17:33:34.589036 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:34.589004 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-988qn_d06a057d-92d0-43c3-bd8c-487f8ab1e22b/node-ca/0.log" Jul 20 17:33:35.589603 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:35.589576 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zzb74_29a0099a-00f5-4c62-ad2f-b340fd793176/istio-proxy/0.log" Jul 20 17:33:35.719781 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:35.719744 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-brv2p_cfea2583-c0b7-4473-933c-02d489624373/discovery/0.log" Jul 20 17:33:35.794785 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:35.794750 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/0.log" Jul 20 17:33:35.820347 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:35.820297 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/1.log" Jul 20 17:33:35.918189 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:35.918158 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d855b796b-mp8tb_2d29f98c-cfe7-4be3-b9f7-3980bc6f718e/router/0.log" Jul 20 17:33:36.647933 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:36.647904 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k7gcx_61557d82-0ea0-426a-b5ee-8a24ad219788/serve-healthcheck-canary/0.log" Jul 20 17:33:37.144988 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:37.144957 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-dl6bh_e74d356e-e508-432c-a37c-e274dcd195b7/insights-operator/0.log" Jul 20 17:33:37.145551 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:37.145533 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-dl6bh_e74d356e-e508-432c-a37c-e274dcd195b7/insights-operator/1.log" Jul 20 17:33:37.338898 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:37.338864 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xrd4w_c3457581-df8a-483d-b5ef-7af9b1e5d4b9/kube-rbac-proxy/0.log" Jul 20 17:33:37.362232 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:37.362199 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xrd4w_c3457581-df8a-483d-b5ef-7af9b1e5d4b9/exporter/0.log" Jul 20 17:33:37.385383 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:37.385352 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xrd4w_c3457581-df8a-483d-b5ef-7af9b1e5d4b9/extractor/0.log" Jul 20 17:33:38.889860 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:38.889830 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z68mp/perf-node-gather-daemonset-8rr6z" Jul 20 17:33:40.230146 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:40.230111 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-7jsnt_eeb0e651-9ed6-4579-bb42-58b549be40d1/manager/0.log" Jul 20 17:33:40.249598 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:40.249573 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76b8cd99f7-7jsnt_eeb0e651-9ed6-4579-bb42-58b549be40d1/copy-manifests/0.log" Jul 20 17:33:41.948757 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:41.948729 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86568699db-l5zbj_6784b9bd-7e60-4597-8b18-291d09216022/manager/0.log" Jul 20 17:33:42.119438 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:42.119407 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zzb74_29a0099a-00f5-4c62-ad2f-b340fd793176/istio-proxy/0.log" Jul 20 17:33:42.242612 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:42.242515 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-brv2p_cfea2583-c0b7-4473-933c-02d489624373/discovery/0.log" Jul 20 17:33:42.290119 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:42.290076 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/0.log" Jul 20 17:33:42.319258 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:42.319223 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/1.log" Jul 20 17:33:42.392926 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:42.392897 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d855b796b-mp8tb_2d29f98c-cfe7-4be3-b9f7-3980bc6f718e/router/0.log" Jul 20 17:33:43.645838 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:43.645810 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zzb74_29a0099a-00f5-4c62-ad2f-b340fd793176/istio-proxy/0.log" Jul 20 17:33:43.771790 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:43.771747 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-brv2p_cfea2583-c0b7-4473-933c-02d489624373/discovery/0.log" Jul 20 17:33:43.842648 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:43.842618 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/0.log" Jul 20 17:33:43.868645 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:43.868615 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-px57d_aaff3519-18e0-46e8-93f0-d1f8503fb069/istio-proxy/1.log" Jul 20 17:33:43.948467 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:43.948379 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d855b796b-mp8tb_2d29f98c-cfe7-4be3-b9f7-3980bc6f718e/router/0.log" Jul 20 17:33:47.397231 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:47.397201 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-4hmr6_f09b4f5f-869c-422e-99e6-4b0df74d6733/kube-storage-version-migrator-operator/1.log" Jul 20 17:33:47.398077 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:47.398059 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-4hmr6_f09b4f5f-869c-422e-99e6-4b0df74d6733/kube-storage-version-migrator-operator/0.log" Jul 20 17:33:48.670956 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.670924 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/kube-multus-additional-cni-plugins/0.log" Jul 20 17:33:48.692039 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.692006 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/egress-router-binary-copy/0.log" Jul 20 17:33:48.717684 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.717655 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/cni-plugins/0.log" Jul 20 17:33:48.740456 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.740427 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/bond-cni-plugin/0.log" Jul 20 17:33:48.762379 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.762352 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/routeoverride-cni/0.log" Jul 20 17:33:48.783652 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.783626 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/whereabouts-cni-bincopy/0.log" Jul 20 17:33:48.808618 ip-10-0-131-206 kubenswrapper[2579]: I0720 17:33:48.808590 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9xmgj_3fa635bd-8080-48a0-814d-9b794456bbe0/whereabouts-cni/0.log"